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1 EXECUTIVE SUMMARY 


1.1 Background 
In 2005, Federal Emergency Management Agency (FEMA) contracted to develop and 
deploy a Logistics Supply Chain Management System (LSCMS) to provide near real- 
time asset and commodity tracking, and to improve the speed, performance and 
accountability for the receipt, inventory, deployment, and distribution of critical disaster 
assets and commodities. 


The application has provided the LSCMS Program the core capability needed to support 
both emergency and non-emergency response operations. 
LSCMS Numerex External Interface is intended to: 


e Provide a periodic feed of location information for FEMA shipments/assets. 

* Combine the location information from multiple sources into a single feed. 
o FEMA Global Positing System (GPS) Transponders 

Partner and/or carriers GPS, cellular or other location feeds. 

Latitude 

Longitude 

Date/Time 


оооо 


1.2 Objective 


The objective of the Quality Assurance (QA) Test Management Team is to functionally 
test and conduct validation to reduce risk and ensure that the system will adequately meet 
the program office’s requirements. 


1.3 Approach 
The QA Independent Verification and Validation (IV&V) Test and Evaluation Report is a 
key document to determine that the LSCMS Numerex and Syncada External Interfaces 
with the FEMA Service-Oriented Architecture (SOA) Global Support System (GSS) are 
ready for final operating capability. This document will identify the test case 
(capabilities to be tested) and the results of the tests. The QA Test Management team is 
responsible for the creation of this document and validating that the external interface 
functionally meets the program office’s requirements. 


The IV&V functional testing of LSCMS Numerex External Interface and the FEMA 
SOA GSS will be accomplished by utilizing LSCMS FieldVision application to track 
transponder identification codes provided by Numerex. This test will consist of entering a 
transponder identification code in the FieldVision application to show the current 
location; as well as, the last six locations that were transmitted from the transponder 
module to Numerex and then processed through the FEMA SOA GSS. 
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This test case will involve LSCMS FieldVision application, transponder identification 
tracking codes and support from the Numerex staff to validate the data feed processed 
through FEMA SOA GSS is the information sent from Numerex. 


As directed by the program office, the Bill of Lading Interface to FEMA SOA GSS 
(Syncada External Interface) will not be included in the LSCMS Release 1.00, This 
interface will be included in a future release. 


1.4 Test Environment 
The IV&V functional testing of the Numerex and Syncada External Interface testing was 
performed at the Brook Road Facility, utilizing the LSCMS FieldVision application that 
is housed in the Integrated Test Environment, within the FEMA Consolidated Test 
Facility. 


2 TEST RESULTS and DETAILS 


The verification results for the Numerex and Syncada External Interface are summarized 
below indicating passed, failed or not tested (if any) between the expected results and actual 
results. Separate test data sheets will be attached as an appendix to this report, if applicable. 


2.1 Test Results 


# # # Not 
Test Cases Passed | Failed | Tested 
2.2 Progression Field Vision Test Cases bh 
2.2.1 Numerex Data Feed to SOA GSS X 
2.2.2 Bill of Lading Interface to SOA GSS (Syncada External Interface) Xx 
TOTALS 1 1 


2.2 Progression Test Case Details 
2.2.1 Numerex Data Feed to SOA GSS 


This test consists of utilizing LSCMS FieldVision application to 
validate the location of the Numerex Transponder modules based on 
transponder identification codes. These codes are used to identify 
shipments previous and current locations. 


Test Description 


Test Date: 10/26/2012 CR/DR/PDCR/SCR: 
Tester: Harry Clark Test Case ID: | QC-0001 
Test Status: Passed Script ID: 


Comment: 


Testing of the Numerex External Interface with FEMA SOA GSS 
was successful. NOTE: During IV&V the geographical map did not 
displayed correctly. Numerex data was validated using data 
confirmation. Reference Section 3 for further details. 
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2.2.2 Bill of Lading Interface to SOA GSS (Syncada External Interface) 
AES Using Syncata the LSCMS will provide Bill of Lading (BOL) 

Че Экер, information to the GSA Transportation system. 

Test Date: CR/DR/PDCR/SCR: 

Tester: Harry Clark Test Case ID: QC-0001 

Test Status: Not Tested Script ID: 
The Syncata portion of the LSCMS was not available for IV&V 

Comment: testing. The program office has decided to include this functionality 
in a future release. 


3 CONCLUSIONS 


The QA Test Management team executed the IV&V functional testing of the connectivity 
between the Numerex external interface and the FEMA SOA GSS utilizing the LSCMS 
FieldVision application to monitor transponder locations. The QA-TM tester found that the 
Numerex Data Feed (xml files that consisted of the physical location of transponder 
modules) was successful in transmitting from Numerex through the FEMA SOA GSS. This 
data was verified by screen shots and reports that were provided by Numerex staff during 
IV&V testing. 


However, during IV&V testing QA-TM tester experienced an issue with the geographical 
map displaying correctly in the FieldVision application. This issue is listed below with an 
explanation of the issue that was reported to the program office. 


High Severity Level 
* Geographical map display 
o When performing a search in the FieldVision application, the tester was able 
to see the transponder icon on the screen, however there was no geographical 
reference of a map to show where the transponder was located. 


To validate this issue additional testing was conducted by the QA-TM tester. These tests 
were conducted using the following FEMA configurations for workstations to validate 
repeatability. 


The workstation configurations that were included in this effort are as follows: 


ə Windows 7 Operating System (OS) and Internet Explorer (IE) 8 
o LSCMS Gold Image - Failed 
o FEMA Standard Gold Image - Failed 
o Integrated Test Environment Image - Failed 


QA-TM tester was able to verify the geographical map image does display successfully 
when using the FEMA Standard Gold Image with XP OS and IE 8, which is not currently 
the FEMA standard nor the FEMA LSCMS Gold Image standard. 
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Based on the [V&V Functional Test results, the QA Test Management Team finds that the 
LSCMS Numerex External Interface with FEMA SOA GSS is functionally capable of being 
deployed to production. 


RISK 


4.1 LSCMS System Testing 


The LSCMS system is a Commercial Off-The-Shelf (COTS) product that has under gone 
numerous configuration requirement changes to allow this software package to meet the 
program office’s requirements and to allow this system to function correctly in the 
DHS/FEMA production environment. IT-QA finds the testing of LSCMS to be inadequate, 
thereby introducing significant risk to the logistics mission. 


LSCMS is a complex system that should be required to undergo extensive IV&V functional 
and performance testing that includes all active modules within this system. By abstaining 
from the rigorous [V&V testing, the risk associated with the deployment into production is 
elevated and the program office is presumed to accept this risk to meet mission 
requirements. 


4.2 External Interface Testing 


The issue with FieldVision map display found during IV&V Functional testing to be 
deployed into production will have a significant impact to the end-user and the program 
office by not having a physical reference point to indicate exact locations or previous 
destinations of the item(s) in transit. 


The work around identified by the program office was to open a second web browser and 
use Google Maps to reference the geographical location from FieldVision on the Google 
Map screen to determine locations of these transponder devices. 


The FEMA IT has identified the problem to be related to a certificate issue. Coordination 
between the program office and FEMA IT has been imitated to remediate the problem. The 
issue with Syncada External Interface will be addressed in the next LSCMS Release. 
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1 EXECUTIVE SUMMARY 


1.1 Background 
The Logistic Program office elected to implement a commercial off-the shelf (COTS) 
product known as the Logistics Supply Chain Management System (LSCMS) to provide 
near real-time asset and commodity tracking, and to improve the speed, performance and 
accountability for the receipt, inventory, deployment, and distribution of critical disaster 
assets and commodities, 


The following modules are included in the LSCMS overall application: 


Distributed Order Management (DOM) supports the initial Customer Order (i.e., 
Request) and Distribution Order (i.e., Order) for assets and the sourcing decision 
by the Supply Chain Managers. DOM replaces eTasker and part of Trading 
Partner Management (TPM). 


Extended Enterprise Management (EEM) supports overall management of Orders, 
Shipments, Inventory at FEMA locations, and Receipt/Shipment at Staging Areas. 
EEM replaces TPM. 


The FieldVision Component within the EEM module supports Geographical 
Information System (GIS) capabilities that will be used to replace FEMA- 
Intelligent Road/Rail Information System (IRRIS) for LSCMS mapping and 
situational awareness. FieldVision leverages the Microsoft Bing commercial 
mapping service for base maps. The data and GIS processing of LSCMS 
shipment, facilities, assets data is done by FieldVision and is not shared with Bing 


Transportation Planning and Execution (TP&E) supports the selection of 
commercial Carriers for FEMA shipments and the collection and analysis of 
Carrier performance data. TP&E interfaces to the GSA’s Transportation 
Management System (TMS) to provide Carrier selection and BOL information, 
and perhaps receive back 3rd Party Payment status. 


Logistics Gateway (LG) provides an Internet portal for Carriers which serves as 
the front end for TP&E. Carriers log into LG to see loads FEMA has put out for 
bids, to submit their bids, and to provide shipment status updates. 


Warehouse Management (WM) supports receiving, inventory, and shipping 
operations at the FEMA Distribution Centers (DCs). WM is an update to the 
current WM used at the DCs. 


Supply Chain Industries (SCI) – the reporting component. 
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1.2 Objective 


The objective of the Quality Assurance (QA) Test Management Team conducted 
performance testing of the Distribution Order Management module in the LSCMS system 
and determine the capability of the application to initiate the Create Order business 
process. 

The performance tests were to validate that the system performs as expected from the 
user/business perspective and functions according to the program office’s order process. 


1.3 Approach 
The QA Independent Verification and Validation (IV&V) Performance Test and 
Evaluation Report is a key document to determine that the LSCMS release 1.00 is ready 
for final operating capability. This document identifies the steps in the performance script 
(capabilities to be tested) and the pass/fail results of the tests. The QA Test Management 
team is responsible for this document and executing tests for verification. The 
performance test consisted of one performance test script, which is made up of 5 
transactions that represent each web page of the application's order entry process. There 
were four load tests executed beginning at100 concurrent users and increasing in 
increments of 100 users until the program office's expectation of 400 concurrent users 
performance is validation. 


1.3.1 Concerns 


This section describes certain areas within the LSCMS application that pose concerns to 
the overall functionality. 


l. In preparation for functional and performance testing, QA-TM encountered several 
issues with user accounts and permissions being active or having the appropriate 
credentials to perform the business process required. Accounts were recreated 
successfully after following appropriate guidelines provided by the developers. 

As a result of multiple schedule changes for the LSCMS configuration, IV&V 

performance testing schedule was forced to shorten the normal time frame. 

3. The LSCMS required Infrastructure configuration changes to the load balancer to 
enable the system to respond successfully at 200 concurrent users. These 
configurations should have been identified and provided to FEMA Configuration 
Management (CM) as an artifact. 

4. Performance Testing was isolated to only one business process within the 
Distribution Order Management module. IV&V Functional testing was only 
conducted on the external interfaces, which resulted in issues with Numerex and the 
Syncada piece being delayed to be tested at a later time. The IV&V functional and 
performance testing efforts should have included a wider range of the overall LSCMS 
application to include all modules. 

Below are list of modules within LSCMS that should have be considered as areas to 
be tested: 


tv 


N 
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a. Handheld devices (Field Scout that experienced issues during previous IV&V 
testing and during SIT and UAT testing of this release) 

b. Transponders (What is the largest number of combined tests that has been 
conducted on these devices.) 

c. Internal and external users connectivity (DHS One Net stability) 

d. Internal and external email alerts (Exchange server capability) 

e. SOA GSS shared with such applications as DAIP and IPAWS-OPEN 
(Enterprise stability) 

f. Syncada external interface complications — not resolved (Ongoing 
connectivity issues and workaround procedures being created) 

g. Extended Enterprise Management (EEM) — 

h. Logistics Gateway (LG) 

i. Warehouse Management (WM) — tracking and trafficking inventory. . 
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1.4 Test Environment 


The IV&V performance testing of the LSCMS was performed at the Brook Road Facility, 
utilizing the LSCMS Distribution Order Management module, Create Order application 
that is housed in the Integrated Test Environment, within the FEMA Consolidated Test 
Facility. To simulate an external user accessing the LSCMS system, QA Test 
Management team executed these series of test scripts using the HP Saas (Service-as-a- 
Support) environment located outside of the FEMA environment. 


2 TEST RESULTS and DETAILS 


The verification results for the LSCMS release 1.00 are summarized below indicating either 
pass or failed (if any) between the system performance requirements supplied by the 
program office and the actual results. 


2.1 Concurrent Users 400 


During the final testing of 400 concurrent users the performance test indicated the 
application and architecture responded successfully at 130 concurrent users with 0 
application errors. Once the performance test exceeded the 130 concurrent users, the 
application continued to experience two errors (one critical and one medium) as indicated 
below. 


* Critical 

502 Bad Gateway (indicating server received an invalid response from a server in the 
environment it accessed to fulfill the request.), which was repeated throughout the course 
of the performance tests. 


/ FEMA 


• Medium 

During performance testing, the number of connections is normally close to the number 
of concurrent users logged in a system. The LSCMS performance tests revealed these 
numbers were substantially different. For the period of a 100 concurrent user test, the 
connect pool showed more than 300 connections. 

NOTE: This issue has been brought to the attention of FEMA Office of Chief 
Information Officer - Architecture, Engineering and Enterprise Services (OCIO-AEES) 
and is being researched. 
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Maximum | Pass Fail 
Seconds "Transactions | Transactions 


20.505 2,278 6,073 


27.36 2,255 Е 


03 Add пет 7.103 12.891 1.458 | 797 
04 Cancel Order 3.963 7165  |14.549 1243 215 
05 Sign Out 12203 [20.359 | 41.461 837 406 


NOTE: Transactions indicate the test step within a business process that was performed as 
part of the performance test. Passed transaction indicates that step in the business process 
was completed successfully. Failed transaction means the virtual user was unable to 
complete that step in the process. Response times are seconds it took for the application to 
respond to transaction (step being performed). 


2.2 Concurrent Users 200 (Re-Test) 


As a result of the errors found in the 400 concurrent user tests, additional performance 
testing was conducted after infrastructure configuration changes/tuning were made by 
FEMA IT. The retest consisted of 200 concurrent users, which are the number users training 
by the program office to utilize the LSCMS application. As a result of this test zero errors 
were encountered by the application. This test did have a medium error. 


• Medium 

During performance testing, the number of connections is normally close to the number 
of concurrent users logged in a system. The LSCMS performance tests revealed these 
numbers were substantially different. For the period of a 200 concurrent user test, the 
connect pool showed more than 450 connections. 

NOTE: This issue has been brought to the attention of FEMA OCIO-AEES and is being 
researched. 
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Transaction Name Minimum | Average | Maximum | Pass Fail 
Seconds | Seconds | Seconds Transactions | Transactions 
01 Log In 5.038 9.021 16.323 1,316 0 
02 Menu Create Order 8.348 1321 20.568 1,316 | 0 
03 Add Item 7.523 12.8 24.288 1,316 | 0 
04 Cancel Order 4.137 6.987 15.122 1,316 | 0 
05 Sign Out 12.069 | 20.632 31.827 1,316 | 0 
NOTE: Transactions indicate the test step within a business process that was performed as 
part of the performance test. Passed transaction indicates that step in the business process 
was completed successfully. Failed transaction means the virtual user was unable to 
complete that step in the process. Response times are seconds it took for the application to 
respond to transaction (step being performed). 
2.3 Test Results 
Test Cases # Passed | # Failed 
2.2 Performance Test 
2.2.1 Cancel Order (400 Concurrent User) x 
2.2.2 Cancel Order (200 Concurrent User – Retest) | X 
TOTALS | 1 1 


2.2.1 Test Script Details — Cancel Order (400 Concurrent User) 


Test Step Description | Log into the LSCMS application. 
Test Date: 1/8/2013 Aveng Ranas 


9.308 


Time (Seconds): 
Tester: Harry Clark | Total Errors: 6,073 
Test Status: Failed | 
502 Bad Gateway error received afier 130 users successfully logged 
Comment: 


into the LSCMS aj 
Step 2 Menu Create Order 
Click on the menu drop down list, select Order Lifecycle tab and click 


lication. 


Test Step Description Create Order link. 
| Test Date: 1/8/2013 tly rs m 13.891 
| Tester: Harry Clark Total Errors: | 23 
| Test Status: Failed | 
Unable to complete an equal number of transactions for this business 
Comment: process due to errors experienced in Step 1. 
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Enter HQ to Region Support in Support field. Enter CP-WTR- 
DRNKNG-STAND in search item and click search button. After the 
window refreshes, select normal from priority drop down list and click 
next button. 

1/8/2013 Average Response 


Test Step Description 


Test Date: Time (Seconds): 12.891 
Tester: Harry Clark | Total Errors: | 797 
Comment: Unable to complete an equal number of transactions. 

4 Cancel Order 


Click Cancel Order button. The Reason window will appear. Select 
Dup Order from the drop down list and enter TEST in the comment 
field. Click Ok. 


1/8/2013 


Test Step Description 


Average Response 
Time (Seconds): 
Tester: Harry Clark Total Errors: 
Test Status: Failed | 

Unable to complete an equal number of transactions for this business 
process due to errors experienced in Step 1. 


Test Date: 


Comment: 


Step 5 Sign Out 


Click on the Sign Out link and click Ok button. Then close web 


Test Step Description | owser, 


Test Date: 1/8213 Average Repone © | 09.359 
Time (Seconds): 
Tester: Harry Clark | Total Errors: | 406 
Test Status: Failed | | 
Unable to complete an equal number of transactions for this business 
process due to errors experienced in Step 1. 


Comment: 


2.2.2 Test Script Details — Cancel Order (200 Concurrent User - Retest) 


Step 1 Log in 
Test Step Description | Log into the LSCMS application. 
Average Response 


Test Date: 1/8/2013 Time (Seconds): 9.021 
Tester: Harry Clark Total Errors: |0 
Test Status: | Passed | 
Comment: | All transactions were successfully completed. 
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Step 2 Menu Create Order 


Test Step Description Соза хела drop down list, select Order Lifecycle tab and click 
1 Average Response 

Test Date: 1/8/2013 Time (Seconds): 1321 

Tester: Harry Clark Total Errors: 0 

Test Status: Passed 


Comment: All transactions were successfully completed. 


Step 3 Add Item 


Enter HQ to Region Support in Support field. Enter CP-WTR- 
DRNKNG-STAND in search item and click search button. After the 
window refreshes, select normal from priority drop down list and click 
next button. 

Average Response 
as Time (Seconds): БЫ 
Harry Clark Total Errors: 0 
Раззей 
All transactions were successfully completed. 


Test Step Description 


Test Date: 


Tester: 
Test Status: 
Comment: 


Step 4 Cancel Order 


Click Cancel Order button. The Reason window will appear. Select 
Dup Order from the drop down list and enter TEST in the comment 


Test Step Description 


| field. Click Ok. 
> Average Response 
Test Date: 1/8/2013 Time (Seconds): 6.987 
Tester: Harry Clark Total Errors: |0 
Test Status: Passed | 
Comment: All transactions were successfully completed. 


Step 5 Sign Out 


Test Step Description Click on the Sign Out link and click Ok button. Then close web 


browser. 
Test Date: 1/8/213 verses Кирин. «| 15:95 
Time (Seconds): - 
Tester: Harry Clark Total Errors: 0 
| Test Status: Passed 
| Comment: All transactions were successfully completed. 
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3 CONCLUSIONS 


The QA Test Management team developed a performance test script that was designed to 
emulate an individual creating an order to be delivered at a designated site in support of a 
disaster and then cancel that order The first test using 100 concurrent users, executed 
successfully with zero application errors and all average transaction response times were 
under the required limit of 5 seconds. The second test using 200 concurrent users (first 
attempt) experienced 502 Bad Gateway errors after reaching 156 concurrent users logged 
into the application. Through monitoring efforts FEMA OCIO AEES identified issues 
between the load balancer and one of the web servers. This web server was removed to 
allow IV&V Performance Testing to continue. Additional testing will be required once a 
resolution is found. To allow time for performance tuning by FEMA AEES and testing by 
QA-TM, the 300 concurrent user test was removed from the original schedule. The third 
tests consisted of 400 concurrent users, executed with a total of 7,514 failed transactions. 
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Following the 400 concurrent user test, FEMA IT AEES found an issue associated with 
health checks between the web server and the load balancer, which would not allow the load 
to be distributed successfully between the two web servers. QA-TM assisted FEMA IT-EI 
System Integration (SI) and IT-QA-Performance Management with performance tuning and 
testing to resolve this issue. This testing effort consisted of 200 concurrent users running for 
30 minutes to verify the load was distributed from the load balancer between each of the 
web servers successfully and that health checks between these web servers were working 
correctly. The results from this test validated infrastructure configuration changes would 
allow the LSCMS system to preform successfully with 200 concurrent users with zero 
application failed transactions. These infrastructure configuration changes will be captured 
by FEMA IT AEES as part of the Configuration Management process. This testing effort 
will continue as resources are available within the FEMA IT organization. 


During the QA IV&V Performance testing, QA-TM tester experienced issues that were 
listed in section 2.1 of this document. These issues, concerns and risks have been brought to 
the attention of the program office and stakeholders. 


Results were presented to the System Owner on the attached addendum A. The System 
Owner understands and accepts the risk(s) associated with the deployment to the production 
environment. Reference Addendum A (Quality Assurance Branch, Test Management Section 
Summary Report) 


IT-QA concurs with the LSCMS program decision to deploy the system presuming the 
Program Office assumes the Performance Risk and remains diligent in support resolution of 
the reasons for the performance failures indicated in testing. 
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4.1 LSCMS System Testing 


The LSCMS system is a Commercial Off-The-Shelf (COTS) product that has under gone 
numerous configuration changes to allow this software package to meet the program office's 
requirements and to allow this system to function correctly in the DHS/FEMA production 
environment. IT-QA finds the testing of LSCMS to be inadequate, thereby introducing 
significant risk to the logistics mission. 


4.2 Environment 
o LSCMS utilizes the FEMA SOA GSS, an enterprise system that is shared with the 
Disaster Assistance Improvement Program (DAIP) and the IPAWS. Currently 
LSCMS does not have an active backup/failover system available within the 
FEMA SOA GSS. 


o The LSCMS performance tested using the ITE environment located at the FEMA 
Consolidated Test Facility (CTF). The LSCMS production system is scheduled to 
be stood up in the FEMA Data Center 2 location, which uses a different load 
balancer manufacturer than what is used in the CTF-ITE. 


4.3 Connections 
The number of connections normally captured during a performance test, is slightly 
higher than the number of concurrent users logged into an application. The LSCMS 
performance tests revealed substantially higher number of connections. For the period of 
a 200 concurrent user test, the connect pool showed more than 450 connections. 
NOTE: This issue has been brought to the attention of FEMA IT and is being researched. 


4.4 Errors 
Once the test exceeded 130 concurrent users, the next virtual user of the application 
would return 502 Bad Gateway errors. This error was repeated throughout the course of 
the performance test. 
NOTE: Concurrent User accessibility improved following changes made by FEMA IT . 


5 RECOMMENDATIONS 


LSCMS is a complex system that should be required to undergo extensive IV&V functional 
and performance testing that includes all active modules within this system. By abstaining 
from the rigorous IV&V testing, the risk associated with the deployment into production is 
elevated and the program office must accept this risk to meet mission requirements. 


À FEMA 


The risk of system performance indicated in 2.1 of this report is to some extend mitigated by 
the fact that the number of concurrent users will initially be fewer than 100. 
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However, performance failure must be addressed quickly. Even through the initial number 
of users is expected to be fewer then the threshold of performance degradation indicated in 
testing, the number of users will grow to exceed that threshold. 


6 DOCUMENT SIGN-OFF 


The final signatures on the LSCMS release 1.0 QA IV&V Performance Test and Evaluation 
Report are as follows: 


cu ыж <a pw Р 


John Cormack 
QA Branch Chief 


; Date: 22/2/220/ 3 
Michelle Fletchér 


QA Test Management Section Chief 


Date: feb %, 20/3 


10 


į FEMA 


QA IV&V Performance Test and Evaluation Report 


Addendum A 


S FEMA 


1.0 PERFORMANCE TESI SUMMARY 
The purpose of this documcnt is te provide a summary report of tho IV&V Performance Test 
that was conducted # the LSCMS Release 1,00 ia the FEMA Берга! Tesi Prvironmen 
(ITE); as well as provide scsaltts, risks and workarounds thal have been descussed wrth the 
program office 
ЧА Test Management ttam completed an Independent Verification and Validation (IV&V) 
Гехолзавсе Vest that consistec fur testing phases. These ter s were based on the LSCMS 
Distribution Orde- Maragerrent, Create Order business process, as identified by the program 
office. This tes! was executed by Quality Assurance Test Management tear, to validate the 
number o° concurrent users in the 1.SCMS application in a virtua) environment. 


2.0 TEST ASSESSENT / RESULTS 
{he program office submatted requirements for the 1 SCMS Distribution Order Management 
(DOM) module to meet а 400 concurrent vse: performance test with the acceptable response 
time vf 15 seconds per web pays under ‘owl, 


The zesults fron the 400 concurrent use: performance text indicated the upplivation and 
architecture rey2onded successfully at 130 concurrent users wi d 0 application emus, Once 
the performance «esi exceeded the 130 concurrent users, the uzolicuüer cüntiaued ic 
experienced errors as indicated below in the nsk portion (3.0) of this document. 


Pertormiar.ce test results tabic from: the 400 concurrent uscrs; 


Max | Passed Бай 
Transactions 


| Transaction 


т 


12405 
14549 


195 Sign Ош — 17 [аазы зт = 
NOIRE Tronsections indicate the test step wit in а business process that was performed as 
part of the performance test. Passed trarsaction indicates that чер in the business process 
was completed successfully. Faileé transaction means the virtual user was unable tc 
complete thal step in the process. Response times are seconds i took for the appacation to 
respond to transaction (Мер being performed). RISKS 
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Quality Assurance Branch 


3,0 RISKS 
* Environments; 

o LSCMS utilizes the FEMA SOA GSS, an enterprise system that is shared with the 
Disaster Assistance Improvement Program (DAIP) and the IPAWS. Currently 
LSCMS does not have an active backup/failover system available within the 
FEMA SOA GSS. 

o The LSCMS performance tested using the ITF environment located at the FEMA 
Consolidated Test Facility (CTF). The LSCMS prcduction system is scheduled to 
be stood up in the FEMA Data Center 2 Jocation, which uses a different load 
balancer manufacturer than what is used in the CTF-ITE. 


* Connections: During performance testing, the number of connections is normally close 
to the number of concurrent users logged in a system. The LSCMS performance tests 
revealed these numbers were substantially different. For the period of a 200 concurrent 
user test, the connect pool showed more than 450 connections. 

NOTE: This issue has been brought to the attention of FEMA IT and is being researched. 


© Errors: For the duration of performance testing. once the test exceeded 130 concurrent 
users, the next virtual user of the application would return 502 Bad Gateway errors. This 
error was repeated throughout the course of the performance test. 
NOTH: Concurrent User quantity was increased as the геѕџй of performance testing and 
IT engineering tuning. 


4.0 WORKAROUND 

The program office has acknowledged the requirement for the LSCMS to perform successfully 
with 400 concurrent users was based on a catastrophic disaster, in which the user volume would 
increase significantly. Currently the program office has a total of 200 users that have completed 
training and are expected to perform these functions. 


Following the official IV&V Performance Test, QA-TM assisted FEMA IT with performance 
testing and tuning activities. These tests were executed to validate configuration changes that 
would allow the LSCMS DOM module to perform successfully at 200 concurrent users with zero 
application failed transactions. This testing effort will continue as resources are available within 
the FEMA IT organization. It is the QA-TM intention to continue support of additional 
performance tuning for the LSCMS application. 
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1 EXECUTIVE SUMMARY 


1.1 Background 


The Logistics Supply Chain Management System (LSCMS) Program office elected to 
implement a commercial off-the-shelf (COTS) product known as the Logistics Supply 
Chain Management System (LSCMS) to provide near real-time asset and commodity 
tracking, and to improve the speed, performance and accountability for the receipt, 
inventory, deployment and distribution of critical disaster assets and commodities. 


This test involves the external interface of the Syncada Bill-Of-Lading (BOL), PayPort 
Express (PPE) with Federal Emergency Management Agency (FEMA) Service-Oriented 
Architecture (SOA) Global Support System (GSS) and interface within the LSCMS Full 
Operating Capability (FOC), 


To support this FOC process, Transportation Planning and Execution (TP&E) will send 
the Citigroup 3rd Party Payment System Processes (3PPS) Syncada application a 
Completed Bill of Lading (CBOL) for each completed carrier shipment. The CBOL will 
itemize all services provided by the carrier, including changes such as wait time or 
diversion, as well as the calculated cost according to the carriers General Services 
Administration (GSA) rates, 3PPS will match the details of the CBOL to the carrier 
invoice(s) prior to releasing payment to the carrier. 


The Citigroup 3PPS service is based on Syncada's commercialization of the PowerTrack 
COTS software which currently supports over 8,500 transportation carriers. Syncada is a 
joint venture between Visa and US Bank to provide transportation payment services to 
“sponsoring banks” and their customers. Citigroup serves as the Syncada service 
“sponsoring bank” for both GSA and FEMA. As such, Citigroup is responsible for 
establishing and managing the customer business relationship with FEMA, while 
Syncada is responsible for operation of the technical service and integration between the 
Syncada system and LSCMS. 


The SOA GSS is a common platform developed by FEMA to provide consistent service 
level agreements (SLA) with respect to security, integrity, availability and scalability 
across multiple business domains and allows the application within the FEMA 
environment to connect to external systems. 


The BOL Interface is used by the LSCMS Integration Framework. Once a Logistic 
Transportation Manager marks a shipment as “complete” in the TP&E user interface, the 
TP&E Module sends CBOL data for the shipment to the LSCMS Integration Framework. 
This interface provides one way communication from LSCMS to SOA GSS, SOA GSS 
sends neither response nor acknowledgement back to LSCMS. 


The Invoice Interchange interface between the SOA GSS and the 3PPS Invoice 
Interchange interface is initiated by SOA GSS. When a CBOL message is received by the 
BOL interface, the SOA GSS transforms the content of BOL message to the Syncada 
Transportation Order and Invoice (STOI) XML format. 


& FEMA 


The SOA GSS initiates the communication by transmitting one invoice in the 
transformed STOI XML format to 3PPS. After receiving the invoice, 3PPS transmits 
transport related status back to SOA GSS. 


Carrier invoices in STOI XML format are transferred from SOA GSS to 3PPS using the 
Invoice Interchange interface. SOA GSS retrieves the carrier invoice in CBOL format, 
written by the LSCMS Integration Framework. It then removes the LSCMS header 
information, transforms the body of the message from LSCMS XML to STOI XML 
format, adds appropriate 3PPS header information, and posts the transformed CBOL 
message to 3PPS. 


QA IV&V Functional Test and Evaluation Report 


1.2 Objective 


The objective of the Quality Assurance (QA) Test Management Team is to functionally 
test and conduct validation to reduce risk and ensure that the system will adequately meet 
the program office's requirements. 


1.3 Approach 

The QA Independent Verification and Validation (IV&V) Test and Evaluation Report is a 
key document to determine that the Syncada BOL / PPE External Interface 

Release 1.00 is ready for final operating capability. This document will identify the test 
cases (capabilities to be tested) and the results of the tests. The QA Test Management 
team is responsible for the creation of this document and validating that the application 
functionally meets the program office's requirements. 


The IV&V functional testing was accomplished by testing the thirteen test cases. Due to 
timeline constrains, IV&V testing was conducted in unison with User Acceptance Testing 
(UAT). The LSCMS 1.00 release testing validated the external interface of Syncada BOL 
/ PayPort Express (PPE) with FEMA Service-Oriented Architecture (SOA) Global 
Support System (GSS) and interface within the Logistics Supply Chain Management 
System (LSCMS) Full Operating Capability (FOC) software solution. IV&V testing 
executed multiple BOL completion transactions to facilitate the following: 


* Validate that data from LSCMS is populating correctly on the Extensible Markup 
Language (XML). 

LSCMS XML is written to SOA GSS 

SOA GSS successfully transforms the data from LSCMS XML to PPE XML 
SOA GSS pushes transformed XML to PPE 

Users are able to perform reconciliation functions on PPE BOL object 


1.4 Test Environment 

The IV&V functional testing of Syncada BOL / PPE External Interface was performed at 
the FEMA Headquarters, Logistic Program Office area, utilizing the Integrated Test 
Environment, within the FEMA Consolidated Test Facility. 
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2 TEST RESULTS and DETAILS 
The verification results for the Syncada BOL / PPE External Interface release 1.00 are 
summarized below indicating pass or fail (if any) between the expected results and actual 
results. Separate test data sheets will be attached as an appendix to this report, if applicable. 


2.1 Test Results 


Test Cases [_#Passed | # Failed 

22 Test Cases 

2.2.1 Dispatch single CBOL message against BOL to SOAGSS 
with all required fields 

2.2.2 Dispatch single CBOL message against BOL to SOAGSS 
with all required fields (Resend CBOL) 

2.2.3 Dispatch CBOL messages against BOL to SOAGSS with 
all required fields populated and other key fields missing 

2.24 Dispatch CBOL messages against BOL to SOAGSS with 
different permissions of required fields populated 

2.2.5 Dispatch CBOL messages against BOL to SOAGSS with 
spot charge populated 

2.2.6 Dispatch CBOL messages against BOL to SOAGSS with 
accessorial charges populated and no spot charge 

2.2.7 Dispatch CBOL messages against BOL to SOAGSS with 
Accessorial charges populated and spot charge populated 

2.2.8 Dispatch CBOL messages against BOL to SOAGSS with 
accessorial charges populated and spot charge populated x 
with different values 

2.2.9 Dispatch single CBOL message against BOL to SOAGSS x 
with single correction notice event logged. | ШР 

2.2.10 Dispatch single CBOL message against BOL to SOAGSS x 
with multiple correction notice events logged. 

2.2.11 Dispatch single CBOL message against BOL to SOAGSS x 
with all possible fields populated. 

2.2.12 Dispatch single CBOL message against BOL to SOAGSS тг x 
with special characters in notes and comment fields. 

2.2.13 Receive a single transformed CBOL message against 
BOL from SOAGSS 


eM] Ms] KR] |] х,у, ж 


TOTALS 13 
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2.2 Test Case Details 


2.2.1 Dispatch single 
required fields 


CBOL message against BOL to SOAGSS with all 


Test Description: 


required fields populated. 


Dispatch single CBOL message against BOL to SOAGSS with all 


Test Date: 


04/23/2013 


required fields 


esend CBOL) 
Dispatch CBOL messages against BOL to SOAGSS with all required 


Test Deserintión: fields populated (Resend CBOL with same information) 

Test Date: 04/23/2013 CR/DR/PDCR/SCR: | 

Tester: Harry Clark Test Case ID: | 

Test Status: Passed Script ID: | 

c : BOL # CS00001009 - 1st test returned an error with the fuel rate 
omment: 


едасш fields populated and other key fields missing. 
Test Date: 04/23/2013 


Tester: 


charge ability to be changed. 2nd test returned successful. 


2.2.3 Dispatch CBOL messages against BOL to SOAGSS with all required 
fields populated and other key fields missin 


Dispatch CBOL messages against BOL to SOAGSS with all required 


Test Status: 


Comment: 


permissions of 


Test Description: 


2.2.4 Dispatch CBOL messages against BOL to SOAGSS with different 


Harry Clark 

Passed | Script ID: ] 

АП fields were populated correctly and fields left blank were received 
blank in Syncada. 


required fields populated 


Dispatch CBOL messages against BOL to SOAGSS with different 
permissions of required fields populated. 


Test Date: 


Tester: 


Test Status: 


04/23/2013 | CR/DR/PDCR/SCR: 
Harry Clark Test Case ID: 
Passed Script ID: | 


Comment: 


Е 


OL# CS0000852 Tester logged into application with a different 
mission and was able to verify the required fields were populated. 
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Dispatch CBOL messages against BOL to SOAGSS with spot charge 
populated. 

04/23/2013 
Harry Clark 


2.2.6 Dispatch CBOL messages against TE to SOAGSS with accessoria] 
charges pop 


Test Description: 


Dispatch CBOL messages agit BOL to SOAGSS with accessorial 


charges populated and no sp EA populated. 
Test Date: 04/25/2013 | CRDRPDCR/SCR:] — — | 


[reser — — S Hany Clark — — [Tercas DE oo 
Test Saak EE Script ID: 

BOL# CS0000658 1st Test received "fault error". Record did not 
successfully transmit through SOA/GSS. SOA LSCMS Transportation 
SVC had to be restarted. No notification was sent of the error, due to 
email configuration - issue resolved from support staff. 2nd Test 
record was routed successfully through. FEMA IT is conducting 
analysis of the record that experienced the issue to avoid future 
incidents. 


Dispatch CBOL messages against BOL to SOAGSS with accessorial 
charges populated and spot charge populated. 


e e | — — 
Passed 

BOL# CS0000854 Tester entered ae data that resulted in Unit 
Price and Extended Price fields to be inactive (grayed out). TPE tester 
was able to return the record from Syncada to LSCMS for corrections. 
After corrections were made to the record from the LSCMS 
application and resubmitted. The fields were populated and enabled 
correctly. 
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2.2.8 Dispatch CBOL messages against BOL to SOAGSS with accessorial 
charges populated and spot charge populated with different values. 


Dispatch CBOL messages against BOL to SOAGSS with accessorial 


Test Description: charges populated and spot charge populated with different values. 
| Test Date: 04/23/2013 CR/DR/PDCR/SCR: 

Tester: Harry Clark Test Case ID: 

Test Status: Passed Script ID: 

Comment: BOL# CS0000853 All fields were populated correctly. 


2.2.9 Dispatch single CBOL message against BOL to SOAGSS with single 
correction notice event logged. 


E ора - - О T 
Test Description: Dispatch single CBOL message against BOL to SOAGSS with single 
correction notice event logged. 
Test Date: [04/23/2013 CR/DR/PDCR/SCR: 
Tester: Harry Clark Test Case ID: 
Test Status: Passed Script ID: 
Comment: BOL# CS0000657 Events log was populated correctly. 


2.2.10 Dispatch single CBOL message against BOL to SOAGSS with multiple 
correction notice events logged. 


Dispatch single CBOL message against BOL to SOAGSS with 


Test Description: multiple correction notice events logged. 
Test Date: 04/23/2013 CR/DR/PDCR/SCR: 
Tester: Harry Clark 


Test Status: 


Comment: 


2.2.11 Dispatch single CBOL message against BOL to SOAGSS with all 


BOL# CS0000858 All corrections were posted correctly in the event 
log 


possible fields populated. 

: Dispatch single CBOL message against BOL to SOAGSS with all 
Церера | possible fields populated: 
Test Date: 04/23/2013 | CR/DR/PDCR/SCR: 
Tester: Harry Clark Test Case ID: ae c —— 7] 
Test Status: Passed Script ID: 
Comment: ВОГ# CS00001012 АП fields were populated successfully. 
2.2.12 Dispatch single CBOL message against BOL to SOAGSS 

nt Dispatch single CBOL message against BOL to SOAGSS with special 
ЖАН ернар: sires wo» and шлу ticas Р 
Test Date: 04/23/2013 | CR/DR/PDCRISCR: | | 
Tester: Harry Clark Test Case ID: | 
Test Status: Passed Script ID: 
BOL# CS0000057 Special Characters in note field and comment 

Comment: 


function successfully. 


QA IV&V Functional Test and Evaluation Report 


2.2.13 Receive a single transformed CBOL message against BOL from 
_SOAGSS 


Test Description: 


Receive a single transformed CBOL message against BOL from 
SOAGSS with accessorial changes populated and spot charge 
populated. Existing invoice cost matches BOL cost. 


[Test Date: | 04/23/2013 | CR/DR/PDCRISCR:| | 
Harry Clark Test Case ID: 
Passed | Script ID: 


Test Status: 


Comment: During UAT testing each test scenario successfully matched existing 


invoice cost to BOL cost. 


3 CONCLUSIONS 


The QA Test Management team and the Logistic System Program Office, in a joint effort 
completed User Acceptance Test (UAT) and IV&V Functional testing of thirteen test cases 
pertaining to observing external interface file transfer processing between sites. Testers from 
both groups found that all test results passed and met the program office’s requirements. 
These tests validated that the Syncada application, Bill-Of-Lading (BOL) and PayPort 
Express (PPE) External Interface functioned according to requirements. 


During UAT/IVV testing the two issues listed below were discovered. These items were 
resolved by FEMA IT during the testing efforts. 
• BOL# CS0000658 Ist Test received "fault error". Record did not successfully 
transmit through SOA/GSS. SOA LSCMS Transportation SVC had to be restarted. 
No notification was sent of the error, due to email configuration - issue resolved 
from support staff. 
© BOL#CS0000854 Tester entered invalid data that resulted in Unit Price and 
Extended Price fields to be disabled (grayed out). TPE tester was able to return the 
record from Syncada to LSCMS for corrections. After the tester corrected the invalid 
characters and routed the record from LSCMS to Syncada, the fields were populated 
and enabled correctly. 


Based on the IV&V Functional Test results, the QA Test Management Team finds that the 
Syncada Bill-Of-Lading (BOL) PayPort Express (PPE) External Interface release 1.00 is 
functionally capable of being deployed to production with minimal risk. 


3.1 Risks 

Schedule Slippage -The program has encountered several schedule changes as a result of 
external interface connection issues between Syncada BOL and PPE with the FEMA 
SOA GSS. These concerns have resulted in delays with the program office completing 
UAT. A result of these delays, UAT and IV&V testing was done in parallel to meet the 
program office delivery date. 
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Section 1.0: Executive Summary 


In April of 2007, the Assistant Administrator of the Logistics Management Directorate (LMD) 
requested an Assessment of the current systems supporting Total Asset Visibility (TAV) by the 
ТАУ Program Management Office (PMO). The purpose of the Assessment was to evaluate 
whether the Logistics Information Management System (LIMS) could support TAV, and replace 
the functionality provided by Manhattan Associates software — Trading Partner Management 
(TPM) and Warehouse Management (WM). The TAV Program Manager (PM) assembled a team 
from SiloSmashers, Inc. and Logistics Management Institute (LMI), Inc. to conduct the 
Assessment. 


The objectives of the Assessment were: 
*  Near-Term - Determine if Manhattan or LIMS is the right solution for near-term operations — 
until the future vision for TAV is defined, planned, funded, and implemented. 


= Long-Term ~ Determine if Manhattan or LIMS provides a foundation for a future vision of 
TAV. 


* Determine an overall recommendation that addresses near-term and long-term considerations 
and their interdependency. 


During the planning stage of the Assessment, the Federal Emergency Management Agency 
(FEMA) Information Technology (IT) office requested the team to include an evaluation of an 
Oracle supply chain solution, since FEMA had already purchased licenses for many of its 
products. In addition, the team decided to include eSuite (eTasker, eFOSA and Single Point 
Ordering Tracking System — or SPOTS), Integrated Response and Recovery Information System 
(IRRIS), and Telecommunications Information Management and Control System (TIMACS), as 
these systems currently support major functions in FEMA’s supply chain. 


The period of the Assessment was May through September 2007 in three phases, shown below: 


Discovery 

* Requirements for Phase | and Phase 2 

* Current systems and process analysis for TAV and eTasker 

* Analysis of current use of LIMS 

* Technical architecture analysis: LIMS, Manhattan TPM and WM, eSuite, and Oracle 
Analysis 

* Evaluation of Manhattan, LIMS, and Oracle Alternatives 

* Gap Analysis for Phase | and Phase 2 

Recommendations 

* Evaluation of alternatives 

* Determination of recommendation 

* Validation of recommendation with stakeholders 

Because FEMA has yet to document the official requirements for TAV, the Assessment Team 
began the Discovery Phase by deriving a set of requirements for the near-term and the long-term 
vision of TAV. The team defined near-term as the next two years — referred to as Phase 1, and 
long-term as beyond two years — referred to as Phase 2. 
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The team derived the requirements based on the functions of the current systems, discussions 
with FEMA staff and management, several TAV Phase 2 diagrams previously presented to senior 
management, and knowledge of industry supply chain best practices. The team documented more 
than 200 requirements (see Appendix A) within fifteen high-level requirement categories (see 
Section 3.0.) The team rated each requirement as to its importance for Phase 1 and for Phase 2. 


The Discovery Phase also included a systems capabilities analysis. The team assessed each of the 
system’s capabilities on a 3-point scale, analogous to the scale used for the requirements. The 
team first examined the functions provided by software currently in production or licensed by 
FEMA, then looked at the potential capabilities if FEMA purchased additional modules or custom 
developed functionality in the future. These capabilities are listed as the systems’ current and 
potential capabilities respectively (see Section 4.0.) 


Also during the Discovery Phase, the team conducted an analysis of FEMA’s supply chain 
processes to assess the use and issues with the current supply chain systems (see Appendix B.) 
The team documented the as-is process flows and outlined issues and recommendations to 
improve the processes. For example, the team discovered that by improving business processes 
FEMA can broaden the scope of its in-transit visibility capability, which today is primarily 
limited to shipments that are sourced from FEMA Distribution Centers (DCs). 


During the Analysis Phase, the team conducted an evaluation of the system capabilities in relation 
to FEMA's requirements for Phase 1 and Phase 2. The capability analysis revealed that only the 
Manhattan and Oracle solutions have the potential to meet the wide range of FEMA requirements 
and provide a foundation for a near-term/Phase | or a long-term/Phase 2 solution. Oracle scored 
the highest, as it provides modules (some currently owned, some that would have to be 
purchased) supporting the full range of requirements. Manhattan scored slightly lower than 
Oracle, because even with additional modules, it would require augmentation with other systems 
to support user requests and property management. 


System capabilities are an important consideration in selecting a TAV solution, but there are 
other critical factors. Therefore, during the Recommendation Phase, the team augmented the 
capabilities analysis with the evaluation of the LIMS, Manhattan, and Oracle alternatives based 
on the following evaluation factors: 


* How well the Phase 1 solution helped prepare FEMA for Phase 2 
* Соѕі of the solution 

= Time until the solution could be implemented 

* Chance of success (the flip side of risk) 

* Software capabilities 

* Political implications 


The team assessed the three alternatives for each of the factors on a scale of 1 (least favorable) to 
10 (most favorable), then weighted factors based on importance to determine the final weighted 
scores for each alternative for Phase 1 and Phase 2 (see Section 5.0.) 


Table | shows the results of the weighted scoring. The Manhattan alternative scored the highest 
for both Phase 1 and Phase 2. Manhattan scored higher than Oracle largely because Manhattan is 
the incumbent, and thus the risk, cost, time, and political scores for the Manhattan solution are 
better than for the Oracle solution. Manhattan scored higher than the LIMS alternative because, 
while the LIMS alternative is the least expensive, it does not provide the range or depth of 
capabilities to support FEMA's requirements. Although LIMS supports FEMA's property 
management function today, it requires substantial custom development to broaden its scope to 
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support the additional supply chain capabilities for TAV. Manhattan is a Commercial Off-The- 
Shelf (COTS) product with already built functionality based on best practices in supply chain 
management. 


Table 1 - Weighted Solution Ratings 
Weighted Rating 


Phase | Alternative 
Prepar: of | 
for Phase 2 | Cost | Time | Success | Capability | Political 


Weights 10% | 20% | 20% 20% 20% | 10% 

Phase 0.9 08 | 16 15 16 07 
1 0.2 14 | 09 0.9 0.8 0.2 
0.7 оз | оз 0.6 17 0.4 


Weights 


Phase 
2 


Based on the evaluation, the Assessment team recommends that FEMA maintain the current 
Manhattan solution for ТАУ (this also includes eSuite for requests, and IRRIS/OrbiTRAX for in- 
transit visibility), There are trade-offs with each option, as reflected in the scores, but when the 
team evaluated and weighed all the factors, the results showed that Manhattan has the best 
potential for support of FEMA ТАУ in both the near-term for Phase 1 and in the long-term for 
Phase 2. 


Given the uncertainty in the long-term/Phase 2 direction for FEMA Logistics, the team also 
recommends that FEMA only enhance the Manhattan solution on a cost- fied, case-by-case 
basis. FEMA should optimize the current implementation with ongoing fine-tuning of the 
systems, and improvements to processes. Then, as FEMA Logistics clarifies the direction for 
ТАУ, if Manhattan is still the appropriate Phase 2 solution, the agency can purchase additional 
modules and major upgrades at that time. 


The team strongly recommends that FEMA perform a detailed review of this assessment to 
validate the requirements, the assigned priorities, and qualitative and quantitative evaluations of 
each solution, to come to a final decision. In addition, the scope of the Assessment only included 
three alternatives — it did not include a survey of all possible solutions. FEMA may wish to 
broaden its evaluation to investigate additional solutions in the future. 


The most concerning aspect of this Assessment for the team, was recommending a solution 
(Manhattan) that is currently not well received at FEMA, and performing below what should be 
expected from a best of breed solution. Therefore, the Assessment team identified the following 
performance barriers and developed recommendations to help FEMA move ТАУ to an optimum 
state (see Section 6.0.) 

Information Technology Support 

* Poor network performance severely impacts the use of TAV 

* The Hosting environment does not adequately support a mission-critical system 

* ТАУ lacks a solid Continuity of Operations Planning (COOP)/Disaster Recovery capability 
* System security decisions need to address business tradeoffs 
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* Lack of integration and data warehousing infrastructure impedes system usability 

* ТАУ IT support is fragmented and lacks accountability 

Current ТАУ Implementation 

= ТАУ is impacted by a lack of standard IT architecture and development methodology 
* Lack of integration and data sharing among existing systems limits effectiveness 

= System administration functions must be optimized to support TAV 


= Automation Support Branch (ASB) support for TAV Help Desk and Testing requires 
improvement 


* FEMA should consider modifying its current integrator strategy to reduce contractor costs 
and improve service levels 

Property Management versus Supply Chain 

= FEMA must clarify and optimize the functions of property management and supply chain 
management 

* There isa high level of duplication between LIMS and TAV 


*  FEMA's accountable property management practices are affecting the overall cost and 
performance of its supply chain 


Business Processes 


= FEMA Logistics lacks a set of managed and measured business processes, integrated between 
Headquarters and the field, and aligned to a set of overarching business objectives 


* Current asset visibility is limited, but could be expanded via new processes 
* Develop processes for reporting during an event to maintain accurate situational awareness 
* Improve the training process by taking an integrated approach across all TAV systems 


Warehouse Management 

* Appropriate use of the WM software will help optimize warehouse receiving 

*  DCs with Joint Field Office (ЈЕО) kits should manage all components of the ЈЕО as kits in 
the Manhattan warehouse management system 

= DCs need to have designated super users for the Manhattan software to understand the system 
and adjust transactions while maintaining data integrity 

* DCs need to use the Manhattan warehouse management system directed picking, which 
would increase the efficiency of limited staffing during disasters 

Most importantly, the TAV program needs management endorsement of a solid roadmap to 

improve TAV over the next year so that FEMA can make the most of its current investment, 

while deciding its future. Major change can occur with program focus on high priority activities 

that bring the greatest value in the near-term. 
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Section 2.0: Introduction 


2.1 Background 


FEMA has invested in multiple systems to support its unique inventory and supply chain 
management needs. The systems include LIMS, developed by FEMA; TPM and WM by 
Manhattan Associates; eTasker, which was developed as a customized application; and IRRIS, a 
Government Off-the-Shelf (GOTS) product. While each of these systems addresses specific 
portions of FEMA’s supply chain, the implementation of multiple systems has potentially 
resulted in duplicative efforts. 


The initial LIMS was built to support the Property Management Division (PMD) at FEMA 
Logistics. LIMS was designed as an automated inventory control and property management 
system in support of disaster response efforts. The initial LIMS system featured supply 
provisioning, built-in requisitioning, processing, maintenance, readiness and financial features in 
а classified environment. On March 11, 1995, LIMS was named the official agency automated 
system for property management. LIMS continued to evolve through 2003 when LIMS III was 
released. LIMS III was a major architectural upgrade of LIMS, but only included the property 
management functionality. Although requirements were gathered, the implementation did not 
include the supply chain management components. LIMS III is the version of LIMS that is in 
production today at FEMA. 


FEMA initiated the initial TAV concept and system at the end of 2004. FEMA Logistics 
understood there was a gap in their ability to track assets once shipped from DCs and determined 
that an automated system would improve information flow. This information, coupled with 
Global Positioning System (GPS) tracking technology, would give FEMA visibility over the 
supply chain, from inventory and order, to fulfillment and shipping. The software solution by 
Manhattan Associates was selected for a limited rollout of a pilot TAV system at FEMA 
Headquarters, and Regions 4 and 6, including DCs Atlanta and Fort Worth. Hurricane Katrina 
stopped the effort and initial operating capability (IOC) did not occur until May 2006. 


eTasker was also launched in 2006 as part of FEMA Transportation. eTasker is a web-based 
system designed to provide users with an automated, standardized capability associated with the 
request process at FEMA headquarters. It captures the user requirements information sent to the 
Logistics Management Center (LMC), through the sourcing of the required commodity and the 
scheduling of transportation. Two additional systems are in development to replicate the eTasker 
functions at field sites. Those two systems are eFOSA and SPOTS. Together the three systems 
are knows as the eSuite. 


As part of the “new FEMA” transformation in March 2007, ће LMD was reorganized and a new 
Assistant Administrator, Mr. Eric Smith, was appointed. Based on briefings about the numerous 
systems which fall under the directorate, overlaps and duplications of TAV system functionality 
were brought to the forefront. 


In April of 2007, the Mr. Smith requested the TAV PMO conduct an assessment of the current 
systems supporting TAV. Mr. Smith requested that the assessment evaluate whether LIMS could 
be used to support ТАУ, and replace the functionality provided by Manhattan Associates 
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software. The TAV PM assembled a team from SiloSmashers and LMI to conduct the 
assessment. 

The period of the Assessment was May through September 2007 in three phases, shown below: 
Discovery 

= Requirements for Phase | and Phase 2 

* Current systems and process analysis for ТАУ and eTasker 

* Analysis of current use of LIMS 

* Technical architecture analysis: LIMS, Manhattan TPM and WM, eSuite, and Oracle 
Analysis 

* Evaluation of Manhattan, LIMS, and Oracle Alternatives 

= Gap Analysis for Phase 1 and Phase 2 

Recommendations 

* Evaluation of alternatives 

* Determination of recommendation 

* Validation of recommendation with stakeholders 
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2.2 Objectives and Benefits 


TAV Assessment Objectives 


Near Term — Determine if Manhattan or LIMS is the right solution for near-term operations — 
until the future vision for TAV is defined, planned, funded, and implemented. 

Long Term — Determine if Manhattan or LIMS is the right solution for the future vision of 
TAV. 

Determine the overall recommendation that addresses near term and long-term considerations 
and their interdependency. 


TAV Assessment Benefits 


Near and long-term evaluation of whether LIMS can be used instead of current Manhattan 
solution 


Evaluation of the current Manhattan implementation 

Identify opportunities for current system improvements to make it more usable and more cost 
effective 

Documented requirements for Phase | and Phase 2 

Framework for planning Phase 2 — budget/schedule 

Alternatives analysis for OMB Exhibit 300 

Inputs to Enterprise Architecture (EA) documentation required for TAV program 
Documentation of current processes and systems 

Recommendations for improving processes 

Rationalization of property management vs. supply chain management 

Ке елда to address the ongoing issue of overlap and duplication between LIMS 
and TAV 


Near and long-term evaluation of the role of the eSuite applications 
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2.3 Constraints and Assumptions 


This section lists the assumptions and constraints that were presented to FEMA Logistics 
management during the planning process for the TAV Systems Solution Assessment. The 
assumptions and constraints were accepted by FEMA Logistics management. The team modified 
the schedule and deliverables to address the constraints. The team managed to the assumptions 
and did not encounter any issues related to the assumptions for availability of staff or required 
trips to visit locations that utilize the software being evaluated. 


Constraints 

= FEMA IT has asked that the Assessment team examine Oracle Supply Chain, because they 
already have the license. 

* Long-term vision for ТАУ is evolving as a result of the future direction for FEMA LMD, 

* FEMA has not defined the requirements for a ТАУ Phase 2 solution. 

* FEMA has not defined the timeframe of a TAV Phase 2 solution. 


~ The Assessment timeframe must allow for termination of TPM/WM and implementation of 
LIMS by December 31, 2007, if that is the recommendation. 


* The TAV Exhibit 300 must include an operational analysis of the existing operational system. 


* FY '09 Exhibit 300 must include a recommendation for TAV Phase 2 and include costs for 
alternatives. 


* Department of Homeland Security (DHS) and FEMA EA requires concept phase for Phase 2 
completed by September 30, 2007. 


General Assumptions 


* [RRIS and OrbiTRAX will remain in place supporting GPS transponder tracking and in- 
transit visibility. 
* Access to FEMA Resources: 


* Access to technical and user documentation before the assessment begins (LIMS, 
Manhattan, eSuite) 


* Direct access to Manhattan, LIMS, eSuite staff 

* Access to Manhattan, LIMS, eSuite users at Headquarters and selected DCs and regions; 
* Access to external resources 

* Access to Oracle sales, functional, and technical staff 

* Access to a Manhattan commercial customer 

* Access to an Oracle supply chain commercial customer 


Defining requirements for the Phase 1 near-term and Phase 2 long-term timeframes required 
making a number of assumptions about the evolution of FEMA °ѕ logistics strategy — recognizing 
that at this point there is significant uncertainty and many future decisions to be made. The 
assumptions used for the requirements analysis were: 


Phase 1 Assumptions 


* Phase | requirements will be derived via review of the current system implementation. 
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* The focus will be on stabilizing and increasing effectiveness/efficiency of existing operations 
without fundamentally changing the logistics strategy or introducing major new systems or 
systems capabilities. 


Phase 2 Assumptions 


* Phase 2 requirements will be derived based on current understanding of FEMA needs. 

* Phase 2 requirements must reflect FEMA Logistics direction, including potential use of Third 
Party Logistics (3PL) or even Fourth Party Logistics (4PL) service providers. 

* FEMA will increasingly leverage 3PL or even 4PL service providers with correspondingly 
less reliance on organic FEMA Logistics resources. These may include: 
* PL Transportation — given FEMA's new ESF 1 requirements; 


*  3PL Warehousing — either operating government-owned FEMA warehouse facilities or 
contractor owned/operated. (Assume 3PLs for FEMA owned warehouses would not be 
introduced until the existing DC's facilities have been upgraded); and 

=  3/4PL for commodities - FEMA currently employs this model with the US Army Corps of 
Engineers’ (USACE) role for ice and may extend it to private companies (e.g., Wal-Mart) 
or Other Federal Agencies (OFAs) for other commodities such as tarps; 


= Less likely, but possible would be: 
3PL for the request/order process; 


3PL for Total Asset Visibility (e.g.. the Disaster Supply Chain “visibility” part of 
TAV's scope); or even 


APL for all FEMA’s operational logistics responsibilities. 


* There will be less use of intermediate staging areas (MOBs, FOSAs, and pre-positioned local 
warehouses) and more direct shipments further down the supply chain, especially direct to the 
state Point of Distribution (PODs.) 


~ The current DCs, which today have relatively small and/or outdated warehouse facilities will 
be replaced with larger (e.g., 1 million square foot), more modern facilities, sized and 
positioned to support FEMA's requirements. 


* Response to large/major disasters will involve more orchestrated support across DCs, with 
closer DCs shipping material that is needed earlier in the disaster and more distant DCs 
shipping material that can arrive later. 


* The combination of more direct shipping and orchestrated cross-DC response will require a 
much higher level of transportation planning and coordinated execution than in the past. 


* There will be expectations for much higher real-time process and system integration and 
interoperability with partners’ (OFAs, states, vendors, carriers, 3PL/4PL) supply chains. 


= There will be an evolution (versus big bang) to the long-term logistics environment. Thus, 
FEMA will be supporting a mix of organic and 3PL/4PL, staging areas and direct shipments, 
integrated and un-integrated partner supply chains for the next several years. 
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Section 3.0: Requirements 


3.1 Overview 


This section describes FEMA requirements for the TAV systems. As noted in Section 2.3, FEMA 
has yet to document the requirements for TAV, and the team needed a set of requirements against 
which to evaluate the potential TAV solutions. The team based the requirements on the functions 
of the current systems, discussions with FEMA staff and management, several TAV Phase 2 
diagrams previously presented to senior management, and knowledge of industry supply chain 
best practices. The benefit to FEMA is that there are now foundation requirements upon which to 
base further discussions about ТАУ Phase | and Phase 2. The timeframe of the Assessment, 
however, did not include major reviews of the requirements with FEMA stakeholders. The 
requirements would have to be substantially refined and expanded to support the detailed 
requirements statement for ТАУ Phase | and Phase 2 required by EA. 


The team started with the following definition of TAV as a foundation for the requirements: the 
capability to provide FEMA with timely and accurate information on the location, movement, 
quantity, and status of disaster response assets, whether they are in storage, on order, in-transit, 
or delivered to disaster victims. It also includes the capability to act upon that information to 
improve the overall performance of FEMA's disaster supply chain. 


Figure | below provides a summary view of the primary supply chain functions, and depicts their 
relationship to TAV. Each of the functions is further defined in this section. 


Figure 1 - Total Asset Visibility Context 


is Source: n " 
Request y Order Fulfill р Transport POD 


Total Supply Chain Visibility 
Situational Awareness, Historical Reporting and Analysis 


Strategic Planning / Tactical Planning / Operations 


Section 3.0 provides a summary of the requirements for TAV. It also presents ratings of the 
requirements based on the Assessment team’s evaluation of FEMA’s priorities. Appendix A 
provides a comprehensive table of more than 200 requirements, with a definition of each 
requirement and a rating of its importance to FEMA. 
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TAV Requirements Scope 
The scope of the requirements reflects the scope of the current and future TAV, and includes: 


= Operational visibility of all assets and actions (e.g., inventory, requests, orders, shipments, 
and receipts) across the extended disaster supply chain including FEMA, vendors, 3PL/4PL 
service providers, OFAs, states, and non-profit organizations; 

= Coordinated planning for the extended disaster supply chain (e.g., stocking levels/locations, 
response scenarios); 

* Coordinated operations across the extended disaster supply chain (e.g., flow of requests, 
sourcing decisions, orders, shipments): 

= Logistics operations at the FEMA elements of the disaster supply chain, e.g., the LMC, 
Regional Response Coordination Centers (RRCCs), JFOs, DCs, Mobilization Centers 
(MOBs), and Federal Operational Staging Areas (FOSAs); 

* Managing and operating Emergency Support Function (ESF) 1 Transportation for FEMA 
elements of the supply chain and partners that request ESF 1 support; and 

= Complying with and leveraging Federal, DHS, and FEMA policies, guidance, standards, and 
IT investments. 


The assessment looked at requirements and capabilities from both the ТАУ Phase 1, near-term 
perspective (next 2 years), and the TAV Phase 2 long-term (beyond 2 years) perspective. 
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3.2 High Level Requirements 


TAV High Level Requirements Definitions 


The team combined the requirements in Appendix A into the following high level requirements 
shown in Table 2. 


Types of Assets supported 


Table 2 - FEMA TAV High Level Requirements 


Definition 


The capability to support a wide range of asset type: 
commodities, accountable property, teams, etc. 


luding 


Planning 


The capability to support disaster supply chain planning, 
including modeling of scenarios to project inventory 
requirements 


3. Acquisition/contracting The capability to integrate with acquisition/contracting to support 
supply chain planning and operations 

4. Requests The capability to submit, manage, track, and report on all 
requests to FEMA for er assets -- whether from FEMA 
Headquarters, Regions, or states 

5. Sourcing/Orders The capability to support the source selection and ordering of 
а from FEMA, vendors, and partn including the ability to 
manage, track, and report on all ord 

6. Warehouse Management The capability to manage and operate FEMA warehouse facilities 
effectively and efficiently 

7. Transportation The capability to support ESF 1, Transportation, functions and 

| plan and manage transportation for 

8. Staging Area Management | The capability to manage staging areas such as FOSAs, MOBs, 
and regional warehouses 

9. Returns The capability to create, manage, and track the reverse logistics 
process/returns 

10. | Asset/Property The capability to track and account for FEMA accountable 

Management | property. 

11. | In-transit Visibility The capability for FEMA to track and view all inbound and 
outbound shipments while in-transit, and report on near-real time 
status of the shipment 

12. | 3'/4" Party Logistics The capability to integrate 3rd and 4th party logistics providers 

Providers into the overall FEMA Disaster Supply Chain 

13. | Financial Accountability The capability to support financial accounting for disaster assets, 
including the integration of invoice reconciliation information 
with FEMA's master financial system 

14. | Supply Chain Visibility The capability for on-line viewing and reporting of all assets and 
events in the Disaster Supply Chain 

15. | Nonfunctional The capability to support operational requirements (e.g., 


Requirements 


. guidance and regulation 


reliability, scalability) and comply with Federal/DHS/FEMA 


Scoring of Requirements for Phase 1 and Phase 2 


Table 3 identifies the relative importance of TAV systems support for each of the fifteen high 
level requirements. The table is organized as follows: 
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* Columns 2 and 3 reflect the team's assessment of the level of system support needed for each 
requirement in the near-term (Phase 1) and long-term (Phase 2). 


The scale in columns 2 and 3 is based on a combination of how important the function is for 
FEMA’s mission, and how sophisticated/advanced the system support needs to be for that 
function in the given timeframe: 


* Blank: no Logistics system support is required 
. dYehow 0 "nice to have" requirement for some basic level of system support 
* — 2/Light Green: core requirement for at least mid/standard level of system support 


E [MbarkGren: | core requirement for advanced/sophisticated system support 


The “blank” under Planning in Phase | does not mean that FEMA has no need for Logistics 
Planning in the near-term, but that TAV system support for this requirement will realistically be 
addressed in the long-term, during Phase 2, where it is rated as Level 3 — a core requirement for 
advanced support. 


Table 3 - Importance of High Level Requirements for TAV Phase 1 and Phase 2 


nce of Systems Support In 


TAV High Level Requirement MET 
g q 1 Phase 2 


Types of Assets supported 
Planning 


3. Acquisition/ 
Contracting 
Requests 


4. 

5. Sourcing/Orders 

6. Warehouse Management 
7. 

8. 


Transportation 

Staging Area Management 

9. Returns 

10. | Asset/Property Management 

11. | In-transit Visibility 

12. | 3'/4" Party Logistics Providers 

13. | Financial Accountability 

14. | Supply Chain Visibility 

15. | Nonfunctional Requirements (operations 
and compliance) 


Table 4 describes the rationale for the Phase | and Phase 2 importance rankings for each of the 
high level requirements. 


Table 4 - Rationale for Importance Rankings 


Rationale for Importance Ranki 


High Level 
Requirement | Ran Phase 1 Rank Phase 2 
1. Types of 2 = Core need to support wide 2 * Same as Phase 1 
Assets variety of commodities, 
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Rationale for Importance Ranking 


Phase 2 


Management 


warehouse capabilities for 


supported accountable property and other 
items 
* Do not need capabilities to 
support very complex assets 
such as a Department of Defense 
(DoD) weapons system 
2. Planning N/A | * Significant TAV support is 3 * Supply chain planning is a 
impractical in the near-term critical enabler of an integrated 
* Requirements for planning are supply chain and the vision for 
tied to the Logistics Logistics 
transformation and will be * The disaster supply chain 
defined and implemented in the includes a wide variety of types 
Phase 2 timeframes of organizations (FEMA, OFAs, 
states, private industry, non- 
profits, 3PL/4PL) that must be 
integrated, and will add 
complexity to planning 
3. Acquisition/ | | * Need to provide at least 2 * Need industry standard 
Contracting minimal, e.g., ema capabilities for integration 
to request new purchas between supply chain and 
provide purchase orders for use acquisition to see available 
in Receiving contracts, contract status, place 
* Tighter integration has strong orders, purchase orders, and 
dependencies on Purchasing provide feedback on 
system which is beyond our receipts/fulfillment by the 
control and not likely to support vendor 
higher integration in the near * Over the long-term, the 
term purchasing system should be 
able to support this higher level 
of integration 
4. Requests 2 * Core need to robustly capture 2 * Same as Phase | 
and process requests 
* Do not have need for complex 
requests with features such as 
intelligent configuration for 
orders (e.g., make sure all the 
many parts of a computer system 
are included and compatible) or 
sales programs for customer 
orders 
5. Sourcing/ 2 * Core requirement to be able to 3 * In the long-term need full 
Orders process and manage orders integration and relationship 
* In the near-term, still mainly management with partners, 
FEMA-centric, with people vendors, 3PL/4PL providers, and 
doing the analysis of what's best Transportation Management 
" System should provide 
intelligent analysis and decision 
capabilities to help in sourcing 
6.Warehouse |2 * Need industry standard level of | 2 * [n the long-term should leverage 


additional sophisticated features, 
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Rationale for Importance Ranking 


Requirement 


Phase 1 


receipt, inventory and shipment 
processes 

Need warehouse integrated with 
order process 

Do not need highly 
advanced/specialized features 
such as leading edge warehouse 
systems 


Phase 2 


but leading edge features not 
needed by core FEMA mission 


7. 
Transportation 


Need at least some level of 
integration with transportation to 
show what carrier resources are 
assigned to a shipment and 
status 

In the near term robust 
transportation management is 
not practical 

FEMA needs to focus on doing 
the essential basic functions 
needed to support its new ESF 1 
mission 


Sophisticated Transportation 
planning, operations and 
management is key to the long 
term logistics direction for 
FEMA with coordinated 
shipments from multiple DCs 
and direct shipment further down 
stream 

Transportation planning and 
management needs to be tightly 
integrated into supply chain 
planning, ordering, and 
fulfillment 

FEMA needs to robustly support 
its ESF 1 mission 


8. Staging * Managing staging areas is a core * Same as Phase 1 
Area requirement for FEMA since so 
Management much of the total assets flow 
through them 
* Staging areas need effective, 
easy to use tools rather than 
more complex features 
* Staging area tools need to be 
integrated into the overall 
requesVorder/inventory/shipmen 
t processes to provide over 
visibility and support higher 
management 
9. Returns * FEMA needs basic capabilities * In the long-term, planning of 
to track returns and manage what materials to return and 
them through the normal request where they should be integrated 
lo receipt process into the overall supply chain 
planning process 
10. Asset/ The Phase | requirements are * Over the long-term, the TAV 
Property largely based on the current LIMS supply chain systems need to be 
Management basic level of capabilities since integrated into an industry 


property management is not the 
focus of the Assessment 


standard level property 
management system to provide 
an overall material/asset solution 
* FEMA does not have a need for 
sophisticated capabilities to 
manage complex assets such as 
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Phase 1 Rank 


Phase 2 


found in military or industrial 
environments 
11. In-transit 2 * FEMA has a core need for in- = Same as Phase | 
Visi transit visibility of inbound and 
outbound shipments 
* FEMA's requirement is similar 
to that facing other large, 
distributed industry and 
government entities and does not 
require especially sophisticated 
or advanced capabilities 
12. 3'/4"" Party | N/A. | * FEMA's 3PL/4PL direction is * Flexible use of 3PL/4PL/organic 
Logistics still being defined and will not providers is expected to play a 
Providers be in place for the near-term critical part of FEMA's long 
term logistics direction 
* [ntegrating external providers 
into the FEMA and overall 
disaster supply chain will require 
robust and flexible systems 
support 
13. Financial 1 * There are currently no interfaces * In the long-term FEMA should 
Accountability to the financial systems support the normal level of 
* The current financial system integration between supply chain 
may be replaced in the next few and financial systems based on 
years industry and government best 
* Ability to associate funding with practices 
request/orders is needed as is 
some way of tracking MOU/IAA 
requests against funding 
14. Supply 2 * FEMA has a core requirement to * FEMA needs to extend visibility 
Chain provide visibility from request to to all of its partners and suppliers 
Visibility receipt * FEMA needs real-time feedback 
* In the near-term FEMA needs to loops to refine its plans and 
provide this primarily for FEMA operations during events. This 
elements of the overall disaster will provide a comprehensive 
supply chain view of how appropriate the plan 
is and allow on-line refinement 
of operations to address 
problems as they are happening, 
15. Non- 2 * Even though the Phase 1 * The operational and compliance 
functional solution is “temporary” it will be requirements for TAV are not 
Requirements in place for several years and beyond those of typical mission 
(operations thus needs to meet a normal critical operational systems 
and level of operational and 
compliance) compliance requirements 
* The operational and compliance 
requirements for ТАУ are not 
beyond those of typical mission 
critical operational systems 
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Section 4.0: System Capabilities 
4.1 Approach 


This section presents the evaluation of the functional capabilities of existing and potential 
Logistics systems to meet the FEMA requirements described in Section 3.0. This section does not 
evaluate other factors such as cost, time, chance of success, or political implications, addressed in 
Section 5.0. The software capabilities evaluation looks at the functions provided by software 
currently in production or licensed by FEMA, and examines the capabilities of modules that the 
agency could purchase or develop in the future. 


As noted in Section 2.3, Constraints and Assumptions, this assessment does not represent a 
comprehensive review of all potential solutions for TAV. Instead, the primary objective of the 
Assessment was to compare a LIMS solution with a Manhattan solution. However, after 
reviewing the alternatives, the team decided to broaden the evaluation to include the following 
Logistics systems: 

* Oracle - FEMA Information Technology representatives requested Oracle be included, 
because FEMA already owns licenses for many of the Oracle supply chain products. 
However, there was only time/resources for a high level review of Oracle's capabilities; 

* The eSuite — The team included eTasker, eFOSA, and SPOTS because, especially in the 
near-term, eTasker will play an important role for requests in any Manhattan or LIMS based 
solution; 

* IRRIS and OrbiTRAX - The geospatial components of the current TAV, are part of the 
Assessment to show their specialized roles in the overall TAV context. 

* TIMACS - The Assessment includes TIMACS because of its role in the FEMA IT/Telecom 
supply chain, including support at the Disaster Information Systems Clearinghouse (DISC). 

The team evaluated each of the Logistics systems from two perspectives: 

* Current — the current level of capabilities provided in the existing FEMA owned COTS 
products (e.g., Oracle, Manhattan) or FEMA developed custom systems (e.g., LIMS or 
eSuite) - NOTE: Current means owned — not necessarily implemented — FEMA owns 
Oracle licenses that it has not implemented, and likewise, owns licenses for some Manhattan 
components that are not in production today. 

* Potential — the potential level of capabilities that the system could provide in the long-term 
by purchasing or developing additional functionality 

Table 5 below provides the definition for Current and Potential for each of the Logistics 

systems evaluated by the team. 

Table 5 - Definitions of "Current" and "Potential" for Each System 


Software | Current Potential 


Manhattan WM Capabilities of COTS modules 
currently owned, but not necessarily 
used to their full potential 
Manhattan TPM Capabilities of COTS modules 
currently owned, but not necessarily 
used to their full potential 
Manhattan with N/A — Addressed above in 

Ad nal Products “Manhattan WM” and “Manhattan 
beyond WM and ТРМ) | TPM’ 


N/A - addressed in “Manhattan with 
Additional Products” 


N/A - addressed in “Manhattan with 
Additional Products” 


Capabilities of Manhattan if FEMA 
were to purchase additional Manhattan 
COTS products 
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Softwa 
LIMS 


Oracle 


eSuite — eTasker 


Curre 


Capabilities of current custom 
application version in use at FEMA 


Capabilities of COTS modules 
owned by FEMA, but not installed 
(NOTE: although FEMA owns 
Oracle application licenses it has 
not installed or implemented any 
Oracle applications) 

Capabilities of current custom 
application version in use at FEMA 


Prepare. Respond. Deliver. 


Potential 


Capabilities of LIMS with significant 
(but realistic) level of new custom 
software development and deployment 
Capabilities of Oracle, if FEMA were 
to purchase additional Oracle modules 


N/A 


eSuite - eFOSA 


Capabilities of custom application 
currently awaiting deployment at 
FEMA 


N/A 


eSuite - SPOTS 


Capabilities of custom application 
developed for use at FEMA, but not 
| yet fully tested or implemented 


eSuite Future 
Development 


TIMACS 


IRRIS/OrbiTRAX 


N/A 


Capabilities of custom application 
in use at the DISC today (evaluation 
based on a very brief overview 
during the visit to the DISC) 
External service capabilities 
currently utilized at FEMA 


s of eSuite with significant 
(but realistic) level of new custom 
software development and deployment 
(new development may blur lines 
among the existing applications) 

N/A (since TIMACS was not the 
focus of the assessment, it was not 
evaluated with potential 
enhancements) 

Capabilities based on a combination 
of leveraging more of these services 
existing functional capabilities and 
integrating additional information 
sources into their TAV environment 
(e.g., feeds from carriers) 
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4.2 Capability Matrix and Scoring 


The team evaluated each of the Logistics systems for current and potential capabilities to meet 
more than 200 requirements in fifteen high level requirements categories. This detailed 
capabilities matrix is provided in Appendix A. Table 6 is a summary of scoring of each Logistics 
system's capabilities to meet the fifteen high level requirements. 


The first three columns of Table 6 are taken directly from the high level requirements table (See 
Section 3.0). The remaining columns show the capabilities rating for each of the systems. The 
scale for the system capabilities columns is defined analogously to the scale for the 
Requirements: 


" Blank: the system does not support this requirement 
. “Yellow: 0 the system provides some basic support for this requirement 
*  2/Light Green: the system provides mid/standard level of support for this requirement 


LI [зрак Green: | the system provides advanced/sophisticated support for this requirement 


The last row of the table sums the total requirements and capabilities scores in each column. For 
example, the sum of the requirements scores for Phase | is 22, versus a sum of 35 for Phase 2 (as 
expected, Phase 2 has much higher requirements.) Similarly, the LIMS sum of capabilities for the 
current system is 6, which could potentially be raised to 17 by additional LIMS development. 


Table 6 - FEMA TAV High Level Capabilities Matrix 
High Level Requirements High Level System Capabilities 


HE i 
a 

з 3 

Bn 


С – eTasker, 
eFOSA, SPOTS 
T 


Types of Assets 

supported 
2. Planning 
3. Interface Acquisition/ 

Contracting 


„ИШЕ 


= 
3 
3 

| 2 | 
3 


4. Requests [2 [2 {2 | 
5. Sourcing/Orders EMI 2 Е [2 | 
6. Warehouse 
3 S 
Management 


7. Transportation 
8. Staging Area 

Management 
9. Returns 


3 
28 
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High Level Requirements High Level System Capabilities 


Requirement 


C —eTasker, 
eFOSA, SPOTS 


10. Asset/Property 


Management 

11. In-transit Visibility 

12. Support for 3'9/4'" 
Party Logistics 
Providers 

13. Financial Integration 

14. Supply Chain Visibility 

15. Nonfunctional 
Requirements 


Sum of Requirements and | 21 | 35] 
Capabilities 


* TIMACS was only evaluated for the DISC 
C = Current P = Potential 


Table 7 below shows an alternative way of comparing the different systems’ capabilities to meet 
FEMA’s requirements. The first two columns again show the importance of the requirement for 
Phase | and Phase 2. The system capabilities columns now indicate whether the current capability 
meets or exceeds the Phase 1 requirement, and whether the potential capability meets or exceeds 
the Phase 2 requirement. For example, the current capability for Manhattan meets or exceeds 10 
of the 13 Phase 1 requirements, while the potential capability of Manhattan with purchase of 
additional products meets or exceeds 13 of the 15 Phase 2 requirements. 


' Oracle only scores a 2 for potential 3PL/4PL capabilities versus the Phase 2 requirement of 3. In 
discussions with Oracle’s lead supply chain personnel, their suggestion was that any 3PL/4PL providers use 
the Oracle products rather than their own solutions, which would have to be integrated into the overall 
Oracle supply chain solution. In contrast, Manhattan's response was that 3PL/4PLs could use the 
Manhattan products or their own solutions — Manhattan was used to integrating a heterogeneous 
environment. This difference caused us to score Oracle lower than Manhattan on ЗРІ /АРІ. If Oracle is 
selected, this potential issue would have to be addressed during implementation 
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eFOSA, SPOTS 


High Level Requirements 

. Types of Assets ae 
supported 

. Planning 

. Interface Acquisition/ 
Contracting 

. Requests 

. Sourcing/Orders 

. Warehouse Management 

. Transportation 


3 
[2| 
3 
. Staging Area 
Management 
. Returns 
. Asset/Property 


Management 

. In-transit Visibility 

. Support for 3'4/4'^ Party 
Logistics Providers 

. Financial Integration 

. Supply Chain Visibility 

. Nonfunctional 
Requirements 


Number of Requirements/ 
Number of Capabilities which 13 13 |3 
тее! ог exceed the Requirement 


* TIMACS was only evaluated for the DISC 
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Conclusions from the Requirements/Capabilities Matrices: 


Several conclusions are evident from the data in the two matrices (analysis of the more detailed 
data in the requirements and capabilities matrix in Appendix A also supports these high-level 
conclusions). 


* Only the Manhattan and Oracle systems provide support for the bulk of the FEMA 
Phase 1 and Phase 2 requirements: 


* The currently owned Oracle products would support 12 of the 13 Phase 1 requirements. 
Oracle's potential capabilities with purchase of additional modules would support 14 of 
the 15 Phase 2 requirements. 


* The currently owned Manhattan products would support 10 of the 13 Phase 1 
requirements. Manhattan's potential capabilities with purchase of additional modules 
would support 13 of the 15 Phase 2 requirements — all but requests and property 
management. 

* The other systems (LIMS, eSuite, TIMAX, IRRIS/OrbiTRAX) each only provides 
support for a small subset of the requirements. 


* None of these systems currently supports more than 6 out of the 13 Phase 1 requirements, 
or potentially supports more than 5 out of the 15 Phase 2 requirements. Thus, not one of 
these systems provides a real foundation for an overall solution. 

~ The currently owned systems (including Manhattan and Oracle modules) do not 
support all of the Phase 2 high level requirements. 


* Inparticular FEMA would need to purchase either Manhattan's or Oracle's Planning and 
Transportation modules to support Phase 2 requirements for: 
* Planning; 
* Transportation Management; 
*  3PL/4PL Management; and 
*  End-to-End Disaster Supply Chain visibility 


Finally, the team also evaluated the ratings when the systems were combined into integrated TAV 
solutions - including a Current System Solution (Manhattan-based) and a LIMS Solution. NOTE: 
This type of combined solution is not required for Oracle as it provides a total solution. 


* Current System Solution — The existing TAV solution includes Manhattan TPM and WM 
for supply chain management, LIMS for asset management, the eSuite for requests, and 
IRRIS/OrbiTRAX for in-transit visibility. This analysis shows that the current solution 
provides good support for FEMA's requirements. 


e C-12/I3: The current capabilities of this combination of systems support 12 of the 13 
Phase | requirements. The one missing requirement is Transportation, as none of the 
current systems provides this capability. 


+ P— 15/15: The potential capabilities of this combination of systems with purchase of 
additional Manhattan products would support all 15 of the Phase 2 requirements. 


* LIMS Solution — This solution combines LIMS for supply chain and asset management, the 
eSuite for requests, and IRRIS/OrbiTRAX for in-transit visibility. As can be seen from the 
scores below, even a combination of LIMS and eSuite does not provide a real foundation for 
an overall solution. 
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C - 7/13: The current capabilities of this combination of systems support 7 out of 13 
Phase 1 requirements. 


P — 8/15: The potential capabilities of this combination of systems would support 7 out of 
15 of the Phase 2 requirements. 
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4.3 Capabilities by System 


Manhattan 


Manhattan is the only alternative besides Oracle that provides a foundation for the bulk of 
ТЕМА near and long-term supply chain needs. It provides industry leading, high-end supply 
chain solutions, and is designed to work in an extended supply chain environment, integrating 
with partners, suppliers, 3PLs, etc -- all coordinated through Manhattan’s systems. This makes it 
a good fit for FEMA’s requirements. 


However, unlike Oracle, the Manhattan suite does not provide a total solution to Phase 2 
requirements. Manhattan is designed to integrate with other systems that provide: 


* Customer order/request management; 
=  Acquisition/contracting; 

* Property management; and 

* Financial management. 


FEMA would have to address these requirements through other systems. 


Manhattan — Currently Owned 


The currently owned Manhattan software, TPM and WM, provide capabilities to support the core 
requirements of Phase 1, with the exception of requests and property management. When 
Manhattan is combined with eTasker for requests, and LIMS for property management, the 
solution includes the capabilities FEMA requires for a solid Phase 1 solution. 


The implementation of Manhattan supporting FEMA today is, however, severely limited/flawed: 


= TPM/WM's higher end capabilities are not used in the current implementation. For example, 
WM is not being used for directed stows/picks at the DC Atlanta and DC Fort Worth 
warehouses. Instead, FEMA staff determines the locations. Similarly, FEMA is not 
leveraging TPM's capabilities for “on-boarding” vendors/partners so that the ordering 
process and status sharing (orders, inventories, shipments, and receipts) can be conducted on- 
line. Therefore, users see the complexity of Manhattan, without realizing the benefits that 
complexity is designed to deliver; 


* There is no current integration between the TPM/WM modules implemented at FEMA and 
the request (eTasker) or property management (LIMS) systems. This causes extensive 
duplicative manual work; 


* TPM is designed, and normally implemented, to support both Intranet access and Internet 
access to manage interactions with partners, vendors, etc. However, due to FEMA security 
considerations, TPM is currently only accessible from the Intranet (or via Virtual Private 
Network (VPN)) from FEMA computers. FEMA has purchased a product that provides the 
capability to integrate its supply chain with its partners and vendors to provide order status 
and shipment status and asset visibility — but as a result of the current security decisions, it is 
unable to implement this vital capability. This security restriction severely limits TPM’s 
current benefits at FEMA; 

* The current TPM/WM environment is not compliant with DHS/FEMA security, EA, and 
Section 508 requirements; 


© The current FY2005 Certification and Accreditation (C&A) is substantially out of date, 
with a full update required for FY2008. (On the plus side, TPM and WM are the only 
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Logistics systems assessed that are integrated with Integrated Security Access and 
Control (ISAAC) - FEMA’s standard single sign-on environment. ISAAC is used to 
establish locations, users, and user roles. Users log in through ISAAC (FEMA Access 
Management System (FAMS)) and authentication/authorization information is pushed to 
the TPM/WM applications); 

* FEMA/DHS EA has classified the current implementation as a non-compliant system and 
will require this to be addressed in the future; 


* The current TAV Manhattan environment is based on a combination of .Net and Ј2ЕЕ 
with SQL Server (TPM) and Oracle (WM) back ends. Thus, parts are inconsistent with 
FEMA technical direction towards J2EE and Oracle. Manhattan's technology roadmap 
does call for a migration to J2EE and Oracle over time — in particular TPM is scheduled 
to move to Oracle in late 2008/early 2009; 


* The current implementation is based on the 2005 releases of TPM and WM — three 
releases behind Manhattan's current offerings; and 


* Manhattan is currently in discussions with FEMA/DHS on Section 508 compliance. 
Addressing this may require either Manhattan making changes in the core COTS 
products, or FEMA obtaining a waiver to use the applications based on undue burden for 
compliance given the few visually disabled users in warehouse/disaster supply chain 
operations. 


Thus, although the Manhattan solution possesses all the capabilities FEMA requires for Phase 1, 
the agency is not getting the full benefits from the software. 


Manhattan - Potential 


Manhattan has the potential to meet or exceed Phase 2 requirements except for request and 
property management. For those areas, Manhattan has extensive experience integrating with other 
applications such as Enterprise Resource Planning (ERP) systems (Oracle, SAP, etc) other COTS, 
or custom systems. FEMA would have to decide how to address these other requirements in a 
Manhattan based long-term solution. For this assessment, we assumed that FEMA would 
integrate Manhattan with the eSuite to support customer request requirements and with LIMS for 
property management requirements. However, Manhattan would also support other choices — for 
example, integrating Manhattan's supply chain applications with Oracle's request and property 
management packages. 


Achieving Manhattan's potential would require a combination of addressing the limitations in the 
current implementation of TPM/WM, and purchasing and implementing a number of additional 
modules. Table 8 shows the Manhattan products that FEMA currently owns and the additional 
products it would require for full support of Phase 2 requirements. 


One other factor that should be noted: as a leading COTS package, Manhattan is supported by an 
extensive COTS development base, user group, and numerous integrators. However, FEMA is the 
only major Manhattan implementation in the Federal Government. Consequently, there is not an 
extensive network of users or integrators who have implemented Manhattan in a Federal 
environment. 
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Table 8 - Currently Owned and Additional Required Manhattan Products 


Additional Products 
Manhattan Products FEMA Owns Needed 


for Full Capabilities 


Integrated Planning Solutions 
Demand Forecasting 
Demand Forecasting 
Promotion Forecasting 
Replenishment 
| Replenishment Needed 
Vendor Managed Inventory Needed 
tegrated Logistics Solutions 
Distributed Order Management Needed 
Warehouse Management 
Warehouse Management Owned 
Billing Management 
Slotting Optimization 


S 


Labor Management 
Yard Management Needed 
Transportation Management 
Transportation Management Needed 
Audit, Payment & Claims 
Fleet Management Needed 


Transportation Planning Execution 
Transportation Procurement 
Carrier Management Needed 

‘Trading Partner Management Owned EES | 
Reverse Logistics Management Needed 

RFID Solutions 


Performance Management Owned 
(not yet implemented) 


LIMS 


LIMS provides very basic property management capabilities, but does not provide an overall 
foundation for FEMA supply chain requirements. 


The previous LIMS versions supported a wide range of functions including request to receipt and 
even repair functionality. However, funding shortages resulted in a drastically reduced scope of 
capabilities for the current version when it was developed to meet Year 2000 requirements. 


Looking forward, LIMS needs to be either 1) replaced as the FEMA property management 
solution by a best of breed Property/Asset Management application, or 2) supported with a 
reasonable level of investment to enhance its property management capabilities and integrate it 
appropriately with the core supply chain applications, e.g., warehouse management and FEMA's 
financial and acquisition systems. 


LIMS - Current 


LIMS provides very basic property accounting capabilities that have been tailored to the 
intricacies of FEMA’s environment. It does not provide more sophisticated property management 
capabilities such as tracking repair or cost histories for more complex assets, or workflow 
capability, which may become more important in the long-term. 


FEMA TAV Systems Solution Assessment v.1.1 Page 27 
October 15, 2007 
FOR -OFFIGIAL USE-ONL¥ 


(КУ 


ЕЕМА Prepare. Respond. Deliver. 


"ws Total Asset Visibility 


ow ug 
@ 


Beyond property accounting, LIMS does not currently provide а full warehouse management 
solution, For, example, its data model and screens do not include requests, orders, or shipments 
(except to the extent a Property Transfer Report (PTR) is like a Shipment). 


Despite its very low level of funding, LIMS has done initial work in a number of value added 
areas: 


* LIMS has prototyped the capability to show LIMS inventories on Google Maps; 
* LIMS also tracks FEMA GPS transponders associated with LIMS PTRs; 
* LIMS has done initial fielding of a scanner solution for receipts and inventory; and 


* LIMS has done initial design for a satellite-based solution to support field staff that cannot 
get access to LIMS through other solutions. 


There are several security, EA, and technical limitations in the current LIMS implementation: 


* LIMS is not integrated with ISAAC for single sign-on; 

* Because of property management policies, LIMS is designed and implemented as an Intranet 
only application. There is no access from outside the FEMA network (except by VPN) or 
from non-FEMA computers. LIMS user access is currently restricted to FEMA employees, 
primarily Accountable Property Officers (APO) and Inventory Management Specialists 
(IMS), although some warehouse personnel have limited access. Thus, it is not accessible to 
FEMA vendors or ESF partners; 

* FEMA EA classifies LIMS as a non-compliant legacy system; 

* LIMS’ architecture still reflects its 1990s heritage and requires technical reengineering to 
become a viable long-term solution. LIMS is based on Java Applets running on the 
desktop/laptop and an Oracle database backend. The maintenance contractor believes that the 
old design of LIMS causes maintainability issues. They have proposed reengineering the 
LIMS back end and establishing a set of back-end Application Program Interfaces or Services 
that could be used both by the front-end/user interface or by integrating programs. They also 
recommend reexamining the user interface, which still reflects the 1990s design rather than a 
modern web application; and 

* LIMS does not have a COOP site, but is currently investigating establishing a COOP site at 
Denton. 


LIMS - Potential 


With extensive custom development, LIMS could support FEMA long-term mid-level property 
management, warehouse, returns, and basic in-transit visibility requirements. Achieving this 
potential would also require addressing the limitations discussed above. 


LIMS is not a potential solution for other areas of FEMA requirements, such as requests, orders, 
transportation, etc, due to the amount of custom development required. The Assessment team 
concluded that those functions were better served by other systems. The Assessment team 
assumed that eSuite would support requests in a LIMS long-term solution, and that FEMA would 
have to determine whether to build or buy systems to support the other areas. 


As a custom system, FEMA and its contractors must provide all enhancements and support for 
LIMS - there is not a network of developers/integrators who know how to maintain, implement, 
or integrate other systems with LIMS. 


The LIMS support contract is moving from a Verizon subcontractor to a DHS Eagle contract this 
fall. The Statement of Work (not yet released) calls for a several month overlap between the 
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current and new contractor during which time the new Eagle contractor would review and design 
any major new requirements arising from this assessment. 


Oracle 


Oracle is the only system that has the potential to address the full range of identified 
requirements, although this would require the purchase of additional Oracle modules that FEMA 
does not currently own. However, because FEMA has not currently implemented Oracle, and 
would require extensive start-up costs and time, Oracle is not practical as a true near term 
solution. 


Oracle - Current 


The currently owned (but not implemented) Oracle products would meet FEMA's Phase 1 
requirements for the core request to delivery processes of the disaster supply chain. However, this 
would require an 18 month or longer implementation timeframe, including bringing Oracle into 
compliance with DHS/FEMA security guidance (e.g., integrating it with ISAAC for single sign- 
on), and ensuring EA compliance. During this timeframe, the current Manhattan environment 
would have to be maintained. Thus, Oracle is not really a near-term solution. 


Oracle - Potential 


With the purchase of Oracle’s planning and transportation modules, Oracle could support the full 
range of FEMA’s Phase 2 requirements. However. Oracle’s potential would stem from its use as 
an ERP solution for FEMA. An Oracle ERP implementation would include financial 
management, human resources, acquisition/contracting, and property management, in addition to 
the supply chain management components. The advantage for FEMA ТАУ would be that the 
integration points between the disaster supply chain and the other systems would be native to the 
software. At the time of this Assessment, FEMA is not pursuing a full Oracle ERP solution, 
which therefore, diminishes the potential benefits of an Oracle solution. 


Thus, selection of Oracle for ТАУ is a decision that would have implications and dependencies 
well beyond the Logistics Directorate. This may turn into an advantage for an Oracle solution if 
FEMA Chief Financial Officer (CFO) selects Oracle as the long-term financial system — DHS has 
mandated Oracle or SAP for financial management of its components. 


Table 9 shows which Oracle modules FEMA already owns licenses for, and which additional 
modules FEMA would have to purchase to obtain the full capabilities to meet long-term 
requirements. 


As suggested by the very number of modules, implementation of a comprehensive Oracle 
solution — as with any major ERP — is a complex, expensive, multi-year process. However, an 
extensive COTS development base, user community, and numerous integrators with Federal 
experience support Oracle. 


Table 9 - Currently Owned and Additional Required Oracle Modules 


Additional Modules 
Oracle Products FEMA Owns to Fully Meet Phase 2 


Requirements 


Supply Chain 
Product Mgmt 
Product Lifecycle Mgmt 
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Additional Modules 
to Fully Meet Phase 2 
Requirements 


CAD Sharing & Viewing 


Owned 


Product Info Mgmt Hub 


UCCnet Trading Connector 


Project Management 


Product Intelligence 


Collaboration Suite 


Order Management 


Order Management 


Configurator 


Advanced Pricing 


Release Management 


Sales Contracts 


Owned 


iStore, Receivables 


Order Mgmt Intelligence 


Owned 


SC Planning 


Network Optimization 


Demand Management 


Forecasting & Modeling 


Sales & Ops Planning 


Advanced SC Planning 


Constraint Based Optim 


Collaborative Planning 


Inventory Optimiz 


Global Order Promis 


Predictive Trade Planning 


Needed 
Needed 
Needed 
Needed 
Needed 
Needed 

ed 
Needed 
Needed 
Needed 


Deductions & Settlements 


Promotion Optimization 


Planning Intelligence 


Procurement 


Sourcing 


Procurement Contracts 


iProcurement 


Services Procurement 


Purchasing 


Supplier Collaboration 


Advanced Pricing 


Payables 


Purchasing Intelligence 


Logistics 


Inventory Management 


Owned 


Mobile Supply Chain Apps 


Owned 


Warehouse Mgmt 


Owned 


Transportation Mgmt 


Operational Planning 


Forwarding & Brokerage 


Freight Pay, Bill, Claims 


Transportation Sourcing 


Cooperative Routing 


Sensor Based Services 


Owned 
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Additional Modules 
Oracle Products FEMA Owns to Fully Meet Phase 2 


Requirements 


Fulfillment Intelligence Owned 
Asset Lifecycle Mgmt 
Enterprise Asset Mgmt EE | 

Self-Service Work Regs 
Asset Tracking 

Install Base 

Property Management 
Maintenance Intelligence 
Manufacturing 

Production Scheduling 
Discrete Manufacturing 
Flow Manufacturing 
Flow Sequencing _ 


Shop Floor Management 

Process Manufacturing 

Mfg Execution System 

Repetitive Mfg Optimization 

Mobile Supply Chain Apps Owned 

Manufacturing Intelligence Owned 
Service 

TeleService Owned 

iSupport Owned 

Field Service Owned 

Mobile Field Service Owned 

Advanced Scheduler Owned 

Spares Management Owned 

Depot Repair Owned 

Service Contracts Owned 

Interaction Center Owned 

Service Intelligence Owned 

Projects 

Project Management Owned 
Project Collaboration Owned 
Project Resource Mgmt Owned 
Project Costing Owned 
Project Billing Owned 


Project Portfolio Analysis 
Projects Intelligence 
Financial Mgt. 

Financials 

Advanced Collections 
Internet Expenses 
iReceivables 

Treasury 

Internal Controls Manager 
Financial Consolidation Hub 
Financial Services Consolidation 
Hub 


Financial Intelligence ЕЕЕ 
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Oracle Products 


Financials & Sales Analyzers 


FEMA Owns 


Owned 


Human Resources 


Human Resources 


Self-Service HR 
(includes Talent Mgmt) 
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Additional Modules 
to Fully Meet Phase 2 
Requirements 


iRecruitment 


Payroll 


Time and Labor 


Workforce Scheduling 


Advanced Benefits 


Learning Management 


HR Intelligence 


Supporting Products 


Project Resource Mgmt 


Incentive Compensation 


Collaboration Suite 


Portal 


User Productivity Kit 


Tutor for Applications 


Self-Service Tutor 


eSuite 


The eSuite currently includes three systems in various stages of maturity. These systems are 
custom, point solutions, developed to support specific Logistics functions at the request of 


specific user groups at Headquarters or in the field. Their capabilit 


s are currently limited by the 


lack of integration among the eSuite systems, and with the other Logistics systems, in particular 


TPM. 


eSuite - Current 


* Overall eSuite Comments - The eSuite was built by a contractor following FEMA's rapid 
system development process using a leading fourth generation web development platform 
(Cold Fusion) and Microsoft SQL Server database (versus the FEMA standard of Oracle.) As 
such, it is not yet compliant with the full set of DHS/FEMA security, EA, and Section 508 


requirements. 


* ебине will be included in the overall TAV security C&A and EA compliance processes 


during FY2008; 


* Suite is not integrated with ISAAC for single sign-on. Planning for integration of 
eTasker with ISAAC is just starting, and lessons learned will be applied to еЕО$А and 


SPOTS; 


* The applications are designed and fielded as an Intranet only application. There is no 
access from outside the FEMA network, or from non-FEMA computers. This restriction 
limits its usefulness for vendors and partners; 


* eTasker Section 508 compliance has been tested by DHS/FEMA and remediation will 
begin in September and be completed by January 2008. eFOSA and SPOTS will have to 
be tested and any issues addressed by leveraging the changes made for eTasker; and 
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* eSuite’s primary operational environment runs on a shared server at FEMA Headquarters, 
and is maintained and administered by the contractor development staff. This 
environment represents a breach in FEMA IT development policies that must be 
corrected. 


*  eSuite has a live backup/COOP environment at Mount Weather. Per direction of FEMA 
IT, the Mount Weather environment will become the operational site this fall. It is 
unclear if the FEMA Headquarters environment will then be retained as a COOP 
environment. 


* eTasker was fielded in selected regions in FY2006 and training of the other regions was 
completed in the summer of 2007. eTasker provides basic support for requests that require 
involvement or action from FEMA Headquarters. Requests that originate in the regions and 
can be satisfied in the regions do not use eTasker. Thus, eTasker only supports a subset of the 
overall disaster supply chain requests. 


As the name suggests, eTasker is an electronic version of FEMA's long-standing "tasker" 
approach — replacing paper, phone calls, faxes, etc with web screens, basic workflow, and 
flexible reports. eTasker is simple and familiar to anybody who knew the manual "tasker" 
process. This makes it easy for users, and users see eTasker as a major step forward over the 
old manual processes. However, it also means that eTasker does not provide any significant 
new "intelligence", capabilities, or different processes than the old “tasker” approach. 


eTasker requests are cross-referenced to TPM Orders so that TPM records the eTasker 
identifier (ID) and eTasker records the TPM Order ID(s). However, there is no interface 
between the systems — data is manually re-entered by Headquarters staff. 


Although the FEMA Logistics Transportation group initially commissioned eTasker, the 
software provides minimal capabilities for transportation requirements. It routes requests with 
transportation needs to the Transportation group and makes sure basic transportation 
fulfillment information has been entered before the request is released. Users conduct all of 
the transportation functions outside of eTasker through phone, email, and fax to Department 
of Transportation (DOT), General Services Administration (GSA), and GSA schedule 
holders. 


In addition, although eTasker collects data about fulfillment of the requests (shipped or 
received status) most of the field sites where these events happen do not have online update 
access. Instead, Headquarters staff update the status based on emails or calls to/from the field. 
Again, there is no interface to the TPM order system, which also collects the same status 
information, resulting in duplicate steps for personnel. 


eTasker is not integrated with FEMA GPS transponders to show in-transit visibility. Thus, 
users acquire in-transit visibility of eTasker requests by querying on the associated TPM 
Order numbers in TPM or IRRIS. 


*  eFOSA has been developed and tested, and is expected to be fielded in 2007. eFOSA grew 
out of a region/FOSA initiative and provides basic support for FOSA staging area 
requirements — receiving shipments, maintaining inventory of trailers and their contents at the 
FOSA, receiving regions' requests to send material to PODs, and arranging/tracking the 
FOSA's shipments to the PODs. Like eTasker, eFOSA automates existing manual processes 
and is intuitive for experienced FOSA staff to use. 


eFOSA and eTasker, although built on the same platform by the same group. are separate 
applications with separate logons, separate databases and no interface between them. The 


FEMA TAV Systems Solution Assessment v.1.1 Page 33 
October 15, 2007 
TOR-OFFICIALUSE ONLY 


(ОУУ 


FEMA Prepare. Respond. Deliver. 


“psc Total Asset Visibility 


cu 


eFOSA application only provides data on an individual FOSA — Headquarters staff can look 
at any FOSA, but there are no built-in queries or reports that aggregate information from 
multiple FOSAs. This reflects the narrow design goal of automating functions at a FOSA, 
rather than establishing visibility of FOSAs in the overall TAV environment. 


This narrow perspective results in significant capability limitations: 


= FOSA managers cannot see inbound shipment information from eTasker (ог TPM); 


* Neither eTasker nor TPM have visibility of eFOSA receipts, inventory, or shipments to 
the PODs unless the information is manually reentered; and 


* Regions or Headquarters cannot see overall receipts/assets/shipments for all FOSAs in a 
region or for all FOSAs combined. Getting this aggregate information would require 
either running a report for each FOSA and then adding up the numbers, or enhancing the 
application to provide multi-FOSA views. 


* SPOTS - SPOTS has been designed, but development/testing has not been completed and no 
date has been set for rollout. SPOTS is designed as a single point for FEMA Regional orders 
that do not require FEMA Headquarters resources. This includes routine administrative 
support requests, e.g., for supplies at regional sites, as well as disaster- related requests. 


SPOTS provides support for requests, approval/sourcing workflow, and order processing with 
regional finance and contracting. As with eFOSA, SPOTS is designed as a separate 
application built on the eTasker environment. Moreover, like eFOSA, each of the regions is 
separate — the application does not provide reports across multiple regions. However, unlike 
eFOSA, SPOTS does have an interface to eTasker — if a regional request cannot be met 
within the region, SPOTS will generate an eTasker to Headquarters for fulfillment. This is a 
point-to-point interface though — meaning that once the data is sent to eTasker it is no longer 
monitored by SPOTS — instead of a truly integrated single point order system that would 
manage and track all orders, regardless of the source. 


As with eFOSA, while SPOTS serves the needs of a particular community, its lack of 
integration further fragments visibility so that there is no one place to see ALL requests or 
orders. 


eSuite - Potential 


eSuite has the potential to support the request and staging area requirements for FEMA, if the 
issues above are addressed and the individual systems are integrated with themselves and other 
Logistics systems. Beyond that, major custom system development could expand eSuite to 
support basic order/sourcing requirements. However, it would not be realistic to enhance eSuite 
to support the full set of Phase 2 order/sourcing requirements, including tight integration with 
warehouse management systems and on boarding of vendors, partners, and 3PL/4PL service 
providers. This conclusion is due to the level of custom development required to support the full 
scope of Phase 2 


TIMACS 


TIMACS was not originally included in the assessment since the system is part of 
IT/Telecommunications rather than Logistics. However, TIMACS is a key system for the DISC 
and the team recommends that FEMA factor it into future plans. 


TIMACS - Current 
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Requests for DISC laptops or other IT material originate through the TIMACS request 
capabilities (e.g., entered by the IT/Telecom representative at a JFO.) TIMACS order capabilities 
are then used to gather bids from IT/Telecom’s standard suppliers (including the DISC), send the 
order to the winning supplier, and track when the order has been completed. In essence, TIMACS 
performs a combination of the eTasker request functions and the TPM Order functions for the 
IT/Telecom part of FEMA. 


Once the DISC has an order from TIMACS, personnel use eTasker to obtain transportation 
resources through the normal eTasker transportation workflow process. Except for the 
transportation connection, the key request/order processes and visibility for DISC are in 
TIMACS, not eTasker or TPM. 


Based on a brief observation of the screens during a visit to the DISC, TIMACS appears to be a 
fairly old, basic system built on legacy architecture, However, it integrates requests, orders, and 
coordination with suppliers more than any of the existing FEMA Logistics supply chain systems. 


TIMACS - Potential 
TIMACS must be factored into the plans for the Logistics supply chain environment because: 
* Itis an integral part of the DISC's processes; and 


= — Itraises the potential issues around customers (at a ЈЕО, or regular FEMA site) having to use 
different request systems for IT/Telecom assets versus other assets, and as a result, 
complicates FEMA's ability to manage and view all requests and orders for disaster supplies. 


TIMACS provides a process and system path for the request and ordering of disaster assets that is 
outside of the ТАУ systems. As a result, users must learn and work with a separate set of 
procedures and asset visibility is further fragmented. The Assessment team recommends that 
FEMA conduct further analysis to determine if TIMACS can be integrated with the TAV 
systems, or if TIMACS should be replaced by eTasker in the future. 


IRRIS and OrbiTRAX 


The two Geospatial Information Systems (GIS) in the current ТАУ solution - IRRIS and 
OrbiTRAX - play critical roles in providing in-transit visibility. Both are external services for 
which ТАУ is only one of many customers. Thus, they are not dependent on using Manhattan 
software and could be integrated into a LIMS or Oracle-based solution. 


IRRIS/OrbiTRAX - Current 


* OrbiTRAX - OrbiTRAX provides the location information for the GPS transponders 
attached to FEMA trailers and assets such as generators. Every 30 minutes when moving or 
twice a day when stationary, each transponder reports its location through satellites to 
OrbiTRAX. 


The transponder Electronic Serial Numbers (ESNs) and locations are then sent from 
OrbiTRAX to IRRIS. In addition, TPM can be used to query OrbiTRAX for the location of 
specific ESNs and display their location on a map within TPM. The TAV prime contractor, 
Stratix, can also query OrbiTRAX to find the location of specific transponders. 


Because of FEMA security constraints, the interface from OrbiTRAX to TPM is limited to 
HTTPS based queries of specific ESNs when requested by a user. In contrast the interface 
from OrbiTRAX to IRRIS sends the location of all FEMA transponders. 
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OrbiTRAX will be included in the overall ТАУ C&A during FY2008. 


OrbiTRAX has not been certified for Section 508 compliance. They are currently watching 
the Section 508 compliance process for IRRIS and will leverage the lessons learned from that 
experience. 


While FEMA owns the transponders, OrbiTRAX is a subcontractor to Stratix and FEMA 
does not have a direct relationship with the company. 


* IRRIS- IRRIS is the main provider of in-transit location information for TAV. IRRIS 
combines transponder location data from OrbiTRAX and Shipment data from TPM to 
provide in-transit visibility of TAV shipments with basic information about source, 
destination, commodity, and quantity. IRRIS also receives similar data from the Defense 
Logistics Agency (DLA) —as a result, TAV's IRRIS users (FEMA and beginning in summer 
of 2007 the USACE will also have in-transit visibility of DLA disaster shipments. 


IRRIS is implemented as a traditional Internet application and is not integrated with ISAAC 
for single sign-on. The TAV PMO controls accounts to access the TAV data on IRRIS. 
Account holders can access the system from any computer at any location through the 
Internet. 


Independent of FEMA, IRRIS receives feeds of shipment data from many carriers — including 
Landstar, the current prime contractor for ESF 1 support. Access to carrier provided data on 
IRRIS that might be related to FEMA shipments is not controlled by FEMA ~- in fact some of 
it may be public domain. 

There were early discussions about the possibility of establishing an instance of IRRIS inside 
the FEMA firewall so it could be protected in accordance with FEMA security standards, but 
this has not been pursued. Bringing IRRIS into FEMA would raise many issues about access 
to useful DLA or carrier feeds, as well as access to IRRIS by non-FEMA ТАУ users such as 
USACE. 


The FEMA relationship with IRRIS is through a contract with the FEMA GIS office ~ ТАУ 
is just one customer of the GIS office's contract. 


OrbiTRAX/IRRIS - Potential 


FEMA is not taking full advantage of the capabilities of the two GIS systems. Both systems could 
provide additional value through interfaces to other sources/users of disaster supply chain 
information, or leveraging their more sophisticated capabilities: 


* IRRIS receives feeds of locations and shipping information such as Bills of Lading from 
many carriers — including Landstar. Landstar, and IRRIS are currently investigating how this 
information might be used to enhance TAV's in-transit visibility: 

= [RRIS is under contract to DoD, and enhancements to their services developed for DoD are 
available to FEMA, IRRIS' capabilities include numerous value added GIS services such as 
sending an alert when a shipment gets within a specific number of miles of its destination. 
This could be used to alert DCs/FOSAs when incoming shipments are close, or to identify 
when shipments have reached a POD so they can be marked as arrived or even received; 


* The GIS office is investigating other ways of leveraging IRRIS, such as posting Readiness 
Report information to show the locations of FEMA commodity inventory. The GIS office is 
also looking at the possibilities, and security issues, of IRRIS feeding FEMA data to other 
DHS GIS systems such as the iCAV system used by first responders; and 
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* Many carriers use OrbiTRAX to track their shipments and it might be possible to leverage 
their data to enhance FEMA in-transit visibility. 


Realizing this potential will require closer coordination among FEMA ТАУ, the GIS office, and 
Logistics business owners. It will also require opening the OrbiTRAX relationship up beyond the 
current narrow Stratix focus. 
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Section 5.0: Solution Recommendation 


5.1 Overview 


This section presents the Assessment team’s recommendations for а FEMA ТАУ system 
solution. In addition to the capabilities analysis in Section 4.0 and Appendix A, the team assessed 
additional evaluation factors beyond pure systems capabilities. These factors are described below, 
along with the methodology used, a list of the assumptions under which the team worked, 
descriptions of the ТАУ solution alternatives, and evaluations of each solution by factor. 


The team also determined ratings and rankings for each solution. Due to the time constraints of 
the Assessment and the time of year, which was during hurricane season, the rankings and ratings 
were conducted within the team and did not include FEMA personnel. Since the final decision 
must be made by FEMA, the team has provided all of the information that went into the ratings. 
and rankings for FEMA in this Assessment. 


Recommendation Methodology 

The Assessment team followed six steps as it conducted the assessment toward refining the 

conclusions. 

* Identified implementation assumptions for Phase 1 and Phase 2 - The team listed items that 
ranged from funding availability to implementation dates. 


~ Identified solution alternatives - The team concluded that there were three primary solutions 
to consider, with the other systems, TIMACs, eSuite, IRRIS/OrbiTRAX, as components of the 


solutions: 

* Manhattan 
* LIMS 

= Oracle 


* Identified evaluation factors - The team identified six factors in order of importance and 
assigned corresponding weights to quantify the evaluation and remove subjectivity. 

* Conducted a qualitative assessment of each solution - What are the relative costs? What are 
the likely schedules? For each solution alternative, the team stepped through the evaluation 
factors and discussed likely outcomes. 

* Ranked and rated solutions against evaluation factors - The team conducted a sensitivity 
analysis based on the assessment of the solutions. 

* Developed conclusions - Ranking and rating the solutions enabled the team to develop 
supportable conclusions. 


Phase Assumptions 


The team assessed the solutions based upon a set of assumptions under which FEMA needs to 
operate if any of the alternatives are to succeed and achieve the best supply chain solution to meet 
FEMA’s Phase 1 and Phase 2 requirements. Since expectations for a solution should be different 
for near- and long-term results, the team developed a list of assumptions for Phase 1 and Phase 2. 
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Phase 1 Assumptions 


When assessing the Phase 1 options, the assessment team weighed the solutions based on the 
following assumptions 


* Three Year Schedule - FEMA needs to sustain the Phase | capability for a minimum of three 
years while it transforms its logistics and supply chain practices. This schedule includes 
calendar years 2008, 2009, and 2010. 

* Software Functions - FEMA will maintain its current request, order, distribution, and in-transit 
visibility capabilities using existing GOTS and COTS software licenses that it already owns. 
Under this assumption, Phase 1 options will evaluate replacing software in production with 
other options currently under license or to be developed by FEMA. For example, one option 
would require major enhancements to LIMS to provide basic warehousing functionality. 

* Maintain the existing distribution infrastructure - The infrastructure includes FEMA DCs, 
regional warehouses, and FOSAs. FEMA will continue to manage and operate its DCs. 


* Adequate funding - Funding is available to support the development of a Phase 1 Full 
Operational Capability (FOC) as described in each of the three alternative solutions. 


* Maintain LIMS as the property management system - Our evaluation focused on supply 
chain management capabilities and not asset management. To limit discussion on property 
management issues, we assumed LIMS would remain as the FEMA property management 
system for Phase 1. 


* Build and maintain a stable workforce - Thi sment assumes an adequate workforce 
will be sufficiently trained and available to operate the TAV systems and support the required 
supply chain management processes. 

* Efficient information technology (IT) infrastructure — This assessment assumes appropriate 
IT infrastructure for a mission critical, real-time system. This includes reasonable network 
response times, hosting reliability, backup and recovery, and an adequate level of support to 
prevent and resolve issues. 

* Maintain information security - The current implementation of Manhattan software has been 
integrated with the FEMA ISAAC security system. Any system implemented for Phase 1 needs 
to integrate with ISSAC. 


Phase 2 Assumptions 


When as 
following 


ssing the Phase 2 options, the assessment team weighed the solutions based on the 
ssumptions: 


a Phase 2 capabilities may need to commence before calendar year 2011 - Although not yet 
clearly defined, FEMA's Logistics transformation strategy is likely to begin rigorous 
implementation within the next three years. 


* Adequate funding - Funding will be available to support the development of a Phase 2 FOC 
that will support FEMA’s Logistics transformation strategy. Funding will enable the integration 
of the TAV software solution into the long-term strategic operations and technology solution. 


* Implement additional supply chain management capabilities - If not already implemented, 
add functionality to the TAV solution as required. 


= Integrate with Logistics transformation systems - Transformation may include outsourcing 
some logistics functions to industry 3PL and 4PL service providers. This will require eBusiness 
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and service oriented architecture (SOA) interfaces with many systems outside the ТАУ suite of 
systems. 


Integrate with FEMA/DHS enterprise solutions — ТАУ Phase 2 will include interfaces to 
enterprise solutions for acquisition, purchasing, and finance. 


Property Management — As noted above, the Assessment did not focus on asset management 
capabilities. However, several members of the evaluation team have expertise in property 
management software capabilities and recommend that FEMA evaluate other software systems 
that are best of breed for property management in parallel with Phase 2. Regardless of the 
direction FEMA takes for property management, Phase 2 should include integration between 
the supply chain software and the property management software to address the areas of 
duplication. 


Flex with an evolving distribution infrastructure - As FEMA Logistics transforms the 
distribution network, the software must be robust to support the changing environment. As DCs 
and other organizations, i.e., OFAs, Non-Government Organizations (NGOs) and third party 
logistics companies, are relocated, added or deleted from the network, TAV needs to be flexible 
enough to adapt to the new operating environment. 


Adapt as required to a new distribution IT network - With the advent of Logistics 
transformation, the TAV software suite may need to field more sophisticated capabilities to 
support a changing network. This will require flexible, robust systems. 


Evaluation Factors 


The team used its combined experience in the field to develop a set of factors by which to 
evaluate each solution. 


Preparation for Phase 2 — Will the Phase 1 implementation provide a foundation of software 
and experience with modern supply chain factors to support the transition to the more capable 
Phase 2 environment — this factor only applied to Phase |. The better the foundation the Phase 
1 alternative provided for Phase 2, the higher the evaluation value 


Costs - Which solution is likely to result in the lowest costs, not including sunk costs? We 
compared the current costs of maintenance and licenses based on OMB 300 and other data for 
current TAV, and discussed notional acquisition/development estimates. The lower the cost, 
the higher the evaluation value 


Time - Are the implementation schedules achievable? The team assessed current systems" 
statuses and estimated notional implementation and deployment schedules. The shorter the 
implementation/fielding/change management schedules, the higher the evaluation value 


Chance of success - Will FEMA be able to conduct a successful implementation of the 
solution? The team weighed each solution based on observations of the current TAV 
implementations and the groups combined expertise implementing both ERPs and supply chain 
systems. The better the chance of success, the higher the evaluation value 


Software Capabilities — Will the systems provide the capabilities to meet the requirements for 
each Phase? The team referenced the capabilities analysis in Section 4.0 to make its 
assessments. The more capabilities to meet the requirements, the higher the evaluation value 


Political ramifications - Will the solution be politically acceptable across all stakeholders? The 
team discussed that the Katrina-related assessments recommended that FEMA improve its 
ability to track shipments. The team also discussed the high profile status of the current 
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Manhattan-based system, which is currently on the DHS Secretary’s dashboard. The less the 
political ramifications, the higher the evaluation value 


FEMA TAV Systems Solution Assessment v.1.1 
October 15, 2007 


Page 41 


-FOR-OFFICIAL-USE-ONLY- 


SU IS 


lg 


= 


2 
Np See 


Total Asset Visibility 


5.2 Solution Options 
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Based on the requirements (See Section 3.0 and Appendix A), and the systems capabilities (See 
Section 4.0 and Appendix A), the team determined solution sets for the Manhattan, LIMS, and 
Oracle options for FEMA TAV. Each option is described in the tables below. 


Option 1 Manhattan 


Table 10 describes the Manhattan option for Phase | and Phase 2 (new capabilities that must be 
purchased or developed are italicized): 


Table 10 - Manhattan Option 


Requirements Area se 1 Solution Ph olu! 

Planning N/A Manhattan Integrated Planning 
(new?) 

Acquire/Contract for Assets N/A Manhattan TPM/WM + interfaces to 


Acquisition (new) 


Requests eSuite + interfaces among eSuite | eSuite 
systems (new) 
Sourcing/Orders TPM + interface from eSuite TPM + Manhattan Distributed 


(new) 


Order Management (new) 


Warehouse Management 


WM + interface to LIMS (new) 


WM + Manhattan Slotting 
Optimization, Labor Management, 
and Yard Management (new) 


Transportation 


eTasker 


Manhattan Transportation 
Management (new) 


Staging Area Management 


eFOSA, TPM, LIMS + 
interfaces among them (new) 


TPM, LIMS 


Returns eTasker, TPM, WM + interfaces | eTasker, TPM + Manhattan 
among them and with LIMS Integrated Planning (new) 
(new) 
Asset/Property Management | LIMS + interface from WM LIMS (new) + interfaces to Financial 
(new) System, Acquisition System, and Data 
Warehouse (new) 
In-transit Visibility TPM, IRRIS, OrbiTRAX TPM, IRRIS, OrbiTRAX, + visibility 
feeds from partners/vendors/carriers 
(new) 
3rd/4th Party Logistics N/A Interfaces to TPM, Manhattan 
Providers Integrated Planning (new) and 
Transportation (new) 
Financial Accountability LIMS, eSuite, TPM, WM Manhattan, LIMS + interfaces to 


Financial system (new) 


End-to-End Disaster Supply 
Chain Visibility 


eSuite, TPM, IRRIS + interfaces 
among eSuite systems and with 
TPM and IRRIS (new) 


TPM, eSuite, IRRIS + feeds from 
pariners/vendors/carriers (new), and 
interface 10 Data Warehouse (new) 


2 "new" refers to purchase of new modules for Manhattan or Oracle, or custom development of new 
capabilities for LIMS or eSuite, or development of new interfaces. 
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Option 2 LIMS 


Prepare. Respond. Deliver. 


Table 11 provides the recommended LIMS solution for Phase 1 and Phase 2 (new capabilities 
that must be purchased or developed are italicized): 


Require: Area 


Planning 


Table 11 - LIMS Option 


Phase 1 Sol 
N/A 


Phase 2 Solution 
COTS Package - TBD 


Acquire/Contract for Assets 


МА 


eSuite (new) and LIMS (new) + 
interface to Acquisition (new) 


Requests eSuite + integration among eSuite + interface to LIMS (new) 
eSuite systems (new) 
Sourcing/Orders eSuire (new) eSuite (additional new) 
Warehouse Management LIMS (new) + interface to eSuite | LIMS (additional new) 
(new) 
‘Transportation N/A COTS Package - TBD 


Staging Area Management 


eFOSA, LIMS + interface 
between them (new) 


eFOSA and LIMS 


Returns 


eTasker and LIMS 


eTasker and LIMS 


Asset/Property Management 


LIMS 


LIMS (new) 


In-transit Visibility 


eSuite (new) and/or LIMS * 
interface to IRRIS and 


eSuite and/or LIMS, IRRIS, 
OrbiTRAX + visil 


OrbiTRAX (new) | parmers/vendors/carriers (new) 
3rd/4th Party Logistics N/A Custom Interfaces (new) 
Providers 
Financial Accountability N/A eSuite (new), LIMS (new) + custom 


interfaces to Acquisition (new) 


End-to-End Disaster Supply 


Chain Visibility 


eSuite (new), IRRIS 


LIMS (new), eSuite (new), IRRIS + 
feeds from partners/vendors/carriers 
(new), and interface (new) to Data 
Warehouse 


Option 3 Oracle 


Table 12 provides the recommended Oracle solution for Phase 1 and Phase 2 (already purchased 
Oracle modules that have to be implemented are italicized, additional capabilities that must be 
purchased/developed are italicized and underlined): 


Table 12 - Oracle Option 

Requirements Area ase 1 Solution Phase 2 Solution 
Planning N/A Oracle Supply Chain Planning (new) | 
Acquire/Contract for N/A Oracle Procurement (new) or 
Assets interface to other Procurement (new) 
Requests eSuite -> Oracle Order Management 

Oracle Order Management 
Sourcing/Orders Manhattan TPM -> Oracle Order Management + 

Oracle Order Management Procurement (new) 
Warehouse Manhattan WM -> Oracle Warehouse and Inventory 
Management Oracle Warehouse and Inventory Management 

Management 
Transportation М/А Oracle Transportation Management 


(new) 
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Requirements Area Phase 1 Solution Phase 2 Solution 

Staging Area eFOSA, Manhattan TPM, LIMS -> Oracle Mobile Supply Chain 

Management Oracle Mobile Supply Chain Applications + Transportation 
Applications Management (new) 

Returns eTasker, Manhattan TPM, WM -> Oracle Order Management + Supply 
Oracle Order Management Chain Planning (new) 

Asset/Property LIMS -> Oracle Asset Life Cycle Management 

Management Oracle Asset Life Cycle Management 


In-transit Visibility 


Manhattan TPM, IRRIS, OrbiTRAX - 
Oracle, IRRIS, OrbiTRAX + interfaces 
(new) 


Oracle, IRRIS, OrbiTRAX + 
visibility feeds from 


rtners/vendors/carriers (new) 


3rd/4th Party Logistics N/A Interfaces to Oracle (new) 

Providers 

Financial N/A Oracle Financial Management or 

Accountability interface to other Financial 
Management system (new) 

End-to-End Disaster N/A Oracle, IRRIS + feeds from 


Supply Chain Visibility 


partners/vendors/carriers (new), and 
interface to Data Warehouse (new) 
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5.3 Decision Analysis 


To evaluate the options, the team first did a qualitative review of each option against each of the 
five evaluation factors, first for Phase 1 and then for Phase 2. This was a group exercise, prior to 
doing any quantitative ratings of the options. The results of the qualitative evaluations are defined 
below. 


Qualitative Analysis - Phase 1 


Evaluation Factor: Preparation for Phase 2 

* Good foundation for Phase 2 - it builds upon the implementation work that has already been done 
with leading edge COTS package. 

* Already integrated — additional Manhattan packages that would be purchased for Phase 2 are already 
integrated with the TPM and WM packages used in Phase 1. 

* Issues in current implementation of TPM and WM — need to address major issues with current 
implementation to provide a solid foundation for Phase 2. 

* Poor foundation for Phase 2 — LIMS Phase | solution does not introduce advanced supply chain 
capabilities or experience with software applications that support industry standard, much less 
advanced capabilities. 

* No start towards major Phase 2 requirements — Phase 2 requirements for Planning, Transportation, 
or 3PL/4PL are beyond the scope of the LIMS solution; FEMA would have to start from scratch in 
deciding how to support them in Phase 2. 

* Good foundation for Phase 2, but major start up costs and time reduce this benefit — much of 
Phase 1 would be occupied with the initial implementation of Oracle to replace the functions 
supported by Manhattan, LIMS, and eSuite. There would be relatively less time to focus on preparing 
for Phase 2. 

* Clean implementation would be a benefit — the ability to start with a clean implementation for 
Phase | might allow FEMA to correct the mistakes made in the initial Manhattan implementation. 


Evaluation Factor: Cost — Phase 1 

* Cost is high - Manhattan is a complex, relatively expensive solution. However, much of the purchase 
and implementation costs have already been incurred. 

* Current contract is higher in cost than the value FEMA receives — the costs for the overall TAV 
program, which do not directly relate to the application solutions (e.g., transponders, surge staffing, 
etc.) would need to be separated out from the application solution costs and controlled. 

* Costs would be driven by the need to better implement the existing capabilities — this would 
include: rollout of WM to the remaining DCs (licenses, implementation, warehouse setup and 
equipment, such as scan guns); business process reengineering and training to ensure FEMA is using 
the already purchased TPM and WM capabilities effectively; security and EA compliance for eSuite 
(e.g., ISAAC integration; and integration with eSuite and LIMS to mitigate the worst of the duplicate 
processes.) 


Manhattan 
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Evaluation Factor: Cost — Phase 1 


* Lowest cost option — both eSuite and LIMS have a history of relatively inexpensive 
development/maintenance costs. 


= Low cost, but also lower functionality — while significant custom development would be required, it 
would only be feasible, within the time constraints, to custom build the minimal "good enough" level 
of development for each requirement; time to market will be much higher with a custom developed 
option. 

* Costs would include — enhancements to eSuite to replace TPM order functionality and allow 
DCs/field locations to update shipments/receipts; enhancements to LIMS to make it a minimal 
warehouse management system (e.g., LIMS Mobile, cycle counts); minimal LIMS/eSuite interface; 
LIMS and/or eTasker interface to IRRIS to show in-transit visibility; reengineering of LIMS to more 
modern architecture; security compliance, e.g., ISAAC integration, for LIMS and eSuite. 


* Very expensive option ~ Oracle is a complex, comprehensive solution; Oracle solution would require 
replacing in-place solutions (WM, TPM, eSuite, and LIMS.) 

= Licenses for the modules to meet Phase 1 requirements have already been purchased — however, 
the ongoing maintenance fees and high cost of integration offset this benefit. 

* Costs would be driven by need for the full life-cycle — including: plan, design and implement 
(including change management) the Oracle solution from scratch — this is typically many times the 
license fee; continue operations of the status quo environment for at least 18 months before Oracle 
could assume core request to delivery and warehouse management functions. 

= Costs are critically dependent on having a good integrator to drive the implementation — Oracle 

applications require specialized contractors who know the products. Costs can increase significantly if 

the wrong integrator is selected. 


Evaluation Factor: Time — Phase 1 
* Fastest, as the system is already at ТОС — initial startup costs are already incurred; some processes 
and training are in place, but work is needed to ensure user acceptance and full system benefits. 
* FOC is already in progress and budgeted — Estimate is six months to improve initial 
implementation and eighteen months to full integration and vendor on-boarding. 


* By June ‘08 minimal ТОС — it may be possible to implement the basic TAV functionality within 10 
months, however the Manhattan solution would need to be running at the same time. 

* ISAAC integration adds risk — LIMS would need to integrate with ISAAC for user sign-on, which 
will add time and cost to the Phase 1 implementation. 

* Re-architecture of LIMS may be required and will add time — FOC for Phase | includes vendor 
on-boarding and warehouse management functionality — may require major foundation changes for 
LIMS database and application. 

* At least 1+ years to get where you are today with Manhattan — FEMA would need to do a 
requirements/design phase to determine how to implement Oracle. Assume configuration changes and 
other enhancements to the software to meet FEMA requirements. 
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Evaluation Factor: Chance of Success – Phase 1 
* High, because Manhattan is currently in production and slowly gaining acceptance — start-up 
costs and risks are already incurred. 
* Non Section 508 compliant adds risk — current software is not 508 compliant. 


= Integrator adds risk — The current integrator may be challenged to implement software integration 
services required to improve the system to reach Phase 1 FOC. 


= Includes minimum supply chain functionality — LIMS is designed to be a property management 
system and does not offer out-of-the-box supply chain management capabilities, such as order and 
warehouse management. 

* New contractor adds risk — LIMS is being re-competed in the next six months and may be supported 
by a new contractor; regardless, there will be a learning curve before contractor can implement new 
functionality. 

= Unable to support high volumes of transactions — per meetings with LIMS staff - currently, LIMS 
cannot handle a high volume of transactions that might occur during a disaster response; there are 
plans to redesign the system to address this. 

* Low - ERP takes major sustained commitment from management — ERP implementations 
traditionally have a low rate of success; only makes sense if Oracle is selected as ERP. 

* Integration with ISAAC adds risk — ISAAC is not designed for easy integration with COTS; Oracle 
maintains its own version of single sign-on instead of ISAAC. 


Evaluation Factor: Software Capabilities — Phase 1 

* Provides required functionality ~ covers required supply chain functionality; however, FEMA has 
not purchased the Manhattan transportation model which may be needed to handle system 
requirements for ESF 1. 

* Some decrease in usability due to current implementation — Package provides the core 
capabilities, however usability is not optimal due to problems with the way the initial implementation 
was conducted. 

* Best of breed supply chain software — ubiquitous presence across the supply chain management 
industry sector. 

* Duplication can be eliminated by integrating system components or eliminating manual 
processes — current duplication with LIMS, eTasker and spreadsheets can be addressed by business 
processes and basic system integration. 

* Potential to replace eFOSA — Manhattan possesses the capability to sustain FOSA operations, 
including managing trailer yards. 

* Lacks a workflow driven request module — built to integrate with a request module, such as 
eTasker. 

* Has capabilities for inventory management — LIMS ha: 
is the system of record for property. 

* Not set up for transportation — LIMS does not possess any transportation management capabilities 
that would support the ESF 1 mission; this would require major custom development. 

2 * Lacks automation to handle surge staffing — it is labor intensive for functions that are normally 

fr} automated through a supply chain management system. 

= Vendor on-boarding requires customization — this would include exchange of purchase order 
information, advance ship notices, and shipment information. 

* Minimal warehousing functionality -- any increase requires custom development — does not 
handle automated due-ins, directed put-aways and picks, or shipping. 


Manhattan 


capability to store inventory. Currently, it 
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Evaluation Factor: Software Capabilities — Phase 1 

* Effective solution — it covers all the required functionality; however, FEMA would need to invest 
increased costs and time to meet current functionality provided by existing Manhattan-based solution. 

* Naturally integrated across the eBusiness Suite — all components share a common data model and 
workflow processes, which eliminates the need for bulky point to point integration solution. 

* May be less user-friendly than Manhattan — trade-offs in usability are typical with ERPs to cover 
the broad functionality. 

= Only adequate warehouse management module, not as good as "WM" — Oracle is not the best of 
breed for warehouse management; however, FEMA's requirements for warehouse management do not 
require robust capabilities. 


Oracle 


Evaluation Factor: Political — Phase 1 

* Does not require major changes/announcements/justification ~ good message 10 say that current 
solution is working. 

" Cost may be an issue — program schedule and delivery was over-promised to senior management; 
cost of current contract is high compared to what has been delivered; better politically to bring the 
costs down. 

* Field perception is mixed — perception of the system in the field is mixed, but good foundation for 
improvement. 

* Perception will be high sunk costs for no benefit - major commitments were made to senior 
management; poor message to say that FEMA is going back to a home grown system. 

* Going to Pre-Katrina capabilities when raised as major issue — LIMS was raised as an issue in 
several Katrina assessments; this was part of the message to senior management as to why a new 
system was required. 

= Perception will be that FEMA wasted a lot of money- cost of current Manhattan implementation 
will be seen as a waste; Oracle does not provide compelling improvements over Manhattan to justify 
the large investment required to switch to Oracle. 

* Justified if FEMA selects Oracle ERP solution — case could be made for Oracle ONLY if FEMA 
was actively pursuing Oracle for its other enterprise functions. 


Qualitative Analysis - Phase 2 


Evaluation Factor: Cost — Phase 2 
= Less costly than Oracle — while Phase 2 would be a complex, expensive process, the scope would be 
smaller than that of the Oracle alternative. 
* Costs are critically dependent on how well the ТАУ integrator can implement and integrate the 
complex new capabilities — assume reasonable software integration costs; requires integrator with 


= 

p<} software integration expertise; FEMA would need a different integrator or major changes to current 

E: contractual arrangement to bring down relative costs. 

[3 = Costs would be driven by purchase/integration of additional Manhattan modules and the need 

E for systems to fill the gaps left by the Manhattan product line — including: purchase and 
implementation of Manhattan products for Transportation, Planning, and Sourcing optimization; need 
for integration among the different systems to support overall supply chain processes, as well as 
integration with 3PL/4PL providers. Need to integrate eSuite for requests and LIMS for asset 
management as part of the overall Manhattan solution. 
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Evaluation Factor: Cost — Phase 2 


LIMS 


* Lowest cost option ~ based upon the assumption that an alternative would only provide a “good 
enough” solution through a combination of custom development and packages. Alternatively, if the 
team decided to evaluate cost of custom development of a LIMS option to duplicate all of the complex 
supply chain functionality of Oracle or Manhattan, then the cost of LIMS would be much higher, and 
its score much lower. 
Costs are critically dependent on how well the integrator can develop, implement and integrate 
the complex new capabilities — the risk with custom development is that if there are major issues 
with architecture or development then the costs could increase to incorporate rework, Rework is not a 
risk with COTS implementation, as they are already architected. 
= Costs would be driven by a combination of the level of functionality and a combination of 
custom development and COTS purchase and implementation — including: custom development 
of LIMS and eSuite to meet core requirements for requests, orders, warehouse management, staging 
areas, and visibility; and purchase of mid level packages to meet selected specialized requirements 
such as transportation which are impractical to develop. 


Oracle 


| Manhattan 


* Most expensive — due to the need to purchase and implement a number of complex additional 
modules to meet Phase 2 requirements and the relatively limited experience with the Phase 1 solution 
that will just have been implemented. 

* Costs are critically dependent on how well the integrator can implement the complex new 
capabilities — Oracle applications require specialized contractors that know the products. Costs can 
increase significantly if the wrong integrator is selected. 


Of the three solutions, Manhattan would provide the shortest time to achieve Phase 2 
capabilities — this conclusion assumes that the issues with the current Manhattan implementation are 
resolved and that FEMA has employed a reliable, experienced software integration provider. 
Correcting the current implementation would still not take as long as starting from scratch with Oracle 
or custom development with LIMS. 


Custom Solutions require more time than COTS – significant enhancements would be required for 
LIMS for Phase 2. The software life cycle for custom development would increase the time over a 
COTS package, as custom software would require significant software architecture and design, and 
considerably more testing than a COTS package — not to mention the time to code the functionality. 


Longer time to get to Phase 2 than Manhattan — Oracle will take considerable time to reach Phase 

1 functionality — which lengthens the time to get to Phase 2. Even though Oracle should integrate 
quicker than Manhattan, because integration is inherent in an ERP solution, the increase in complexity 
for the ERP over Manhattan would balance the time gain. 


Evaluation Factor: Chance of Success - Phase 2 


* High - Manhattan COTS is specialized for supply chain management — strong tools, widely used. 
* Made for integration with successful track record — commonly integrates with other software 
packages, including SAP and Oracle. 


LIMS 


= Low - takes too long, high cost — Phase 2 would require major custom development. LIMS will not 
provide the Logistics Directorate with a system that supports an increased operations support 
capability, without significant rework. 

= No private sector best practices — commercial software such as Oracle or Manhattan, incorporate 
industry best practices based on their clients” requirements. 
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Evaluation Factor: Chance of Success — Phase 2 
= Low — due to ERP issues and organizational dependencies — ERP installations are typically 
challenging, and require commitment from the entire organization. Implementation of the Oracle 
solution can cause the agency to stumble while IT works through the bugs and users become 
acclimated with the new processes. 


Oracle 


Evaluation Factor: Software Capabilities – Phase 2 
* Effective solution — best of breed and designed to integrate with external providers 
* Transportation module - positioned for ESF 1 ~ with FEMA assuming responsibility for ESF 1 
from the Department of Transportation, the Logistics Directorate will need a way to manage the 
trucking part of the program, 


* Low - LIMS has a low likelihood of providing advanced capability, especially for the pending 


5 transformation environment. Requires all custom development, which will take increased 


* Less usability than Manhattan — due to ERP trade-offs. 

* Transportation module (not currently under license) available - FEMA will need to determine 
whether the Oracle transportation offering meets requirements, considering there are other bolt-on 
transportation packages that may provide the agency with more appropriate solutions. 


Evaluation Factor: Political - Phase 2 


* If Phase 1 issues are addressed, political issues will decrease for Phase 2 — ТАУ is still not 
delivering the value and benefit promised, and now FEMA and DHS management think that TAV will 
be subsumed by 3PL, which is not the case. FEMA needs to resolve these issues to make a case for 
Phase 2. FEMA must adopt a standard development methodology for TAV, and address the major IT 
issues to move TAV to an acceptable operating level. FEMA must communicate and deliver a realistic 
execution path for ТАУ; FEMA must clearly articulate the relationship between ТАУ and 3PL. 

* In Comparison with the other options, Manhattan is the best politically – Phase 2 leverages the 
current investment in Phase 1. 

* Major political ramifications and PR issues — the political issues for LIMS for Phase 1 are the same 
for Phase 2. Assessments after Katrina criticized FEMA for using an outdated system to manage its 
supply chain. To revert to LIMS to support complex Phase 2 functionality would be a difficult sell. 


* Only if sold as part of an ERP implementation — the major issues for Oracle would be that the 
Manhattan solution would be seen as a wasteful mistake from both cost and time. Over time, and in 
the long run, if FEMA does move to an Oracle ERP solution, including Finance, then the case could 
be made to transition to some Oracle components (e.g., Asset Management, Order Management). 
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Quantitative Analysis 


The team then did a ranking of each option for each factor, for each phase, based on the 
qualitative analysis. This was also a group exercise. The results of the ranking are presented in 
Table 13 below. 


Table 13 - Team Ranking, Based on Qualitative Evaluation 


Raw Rating Scores (3 = Best, 1 = Worst 


Option Chance 
Preparation of Solution 
for Phase 2 | Cost | Time | Success | Capability | Political | Average 


1.7 


Next, the team rated each solution, for each factor, by phase. The six key subject matter experts 
of the team conducted the ratings individually. The ratings were then averaged. Each team 
member rated each solution/factor/phase on a scale of 1 to 10, where 10 is the best score and 1 is 
the worst score. In addition, because each of the factors is not of equal importance when 
compared to the other factors, the group weighted the factors and multiplied the ratings for each 
factor by the weighting. The team concluded that Cost, Time, Chance of Success, and Software 
Capability, should be weighted higher than Preparation for Phase 2 and Political implications for 
determining a recommended solution. The results are presented Table 14 below. 


Table 14 - Weighted Solution Ratings 


Phase | Alternative | Chance | 
Prepar: of 


for Phase 2 | Cost те | Success | Сара! 
Weights 10% 20% 
Phase 0.9 0.8 


1 o2 | 14 
07 0.3 


Weights 


Phase 
2 


The table shows that the Manhattan alternative received the highest scores for both Phase 1 and 2: 


* Although both the Oracle and Manhattan solutions rated very high in the capabilities matrix, 
with Oracle slightly higher because of its comprehensive solution, Manhattan rates higher when 
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the other evaluation factors are included. This is because it is the incumbent and is already 
substantially implemented and accepted. This gave it much higher ratings on Cost, Time, 
Chance of Success, and Politics. 


* For Phase 2, the Manhattan and Oracle ratings were closer — although Manhattan still had an 
edge on all factors except Capabilities and Cost. 


= Compared to LIMS, Manhattan rated higher in Phase 1 on all factors except for Cost. In Phase 
2, Manhattan again rated higher than the LIMS alternative on all factors except for Cost. 


The relative rankings of the LIMS and Oracle alternatives varied: 


* For Phase 1, Oracle had marginally higher raw ratings because of better scores in Preparation 
and for Phase 2, Capabilities, and Politics. However, LIMS was higher on Phase 1 rankings and 
on weighted ratings because of the relatively high weighting factors for the Cost, Time, and 
Chance of Success factors, where it scored higher than Oracle. 


* For Phase 2, Oracle scored higher than LIMS on both the raw and weighted scores — scoring 
higher than the LIMS alternative on all factors except for Cost. 
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5.4 Recommendation 


The decision analysis resulted in the team’s recommendation that FEMA maintain and enhance 
the current Manhattan solution for TAV in the near-term. The analysis also revealed that 
Manhattan provides a good foundation for potential future TAV requirements. As the scores 
illustrate in the previous section, there are trade-offs with each option, but when all the factors 
were evaluated and weighed, the team decided that that FEMA has assembled a strong suite of 
software products to meet its near-term needs, and a good foundation on which to base its future 
direction. The team, however, strongly recommends that FEMA perform a detailed review of this 
assessment to validate the requirements, their priorities, and the team’s qualitative and 
quantitative evaluations of each solution, to come to a final decision, 


Manhattan provides a better solution than Oracle because it provides best of breed supply chain 
functionality and it is already in place; FEMA has endured the start-up costs and learning curve, 
and is ready to reap the benefits. Manhattan is a better solution than LIMS because it provides a 
level of functionality that cannot be matched in a custom system. However, as discussed 
previously, the Manhattan software needs to be combined with solutions for requests and property 
management. The Assessment team assumed these requirements would be met by the eSuite and 
LIMS applications respectively. However, FEMA could choose to acquire alternative systems to 
meet these aspects of the Phase 2 requirements. 


The team members understood when the Assessment began, that the TAV solution was not well 
accepted by FEMA staff, and that FEMA was receiving limited value from the systems. This is 
why the team performed an operational analysis of the current systems during the Discovery 
Phase. The team conducted in-depth process analyses of the existing supply chain functions that 
support ТАУ, including trips to the Atlanta and Fort Worth warehouses, interviews with 
commodity managers, LMC staff, and regional staff. In addition, the team interviewed users to 
understand their issues with the current systems, and the development teams to understand their 
perspectives on the problems. 


The Assessment data gathering activities revealed major barriers impeding the effective use of 
FEMA ТАУ, ultimately preventing FEMA from reaping the full benefits of the system. These 
barriers bear no relation to the specific software solutions supporting TAV. They are directly 
attributable to the way FEMA has designed, implemented, and supported this large and critical 
investment. On the positive side, the challenges can be overcome without major software 
investments, as many of the issues can be resolved with consolidation, enhancement, and 
standardization of business processes. Major change can occur with program focus on high 
priority activities that bring the greatest value in the near-term. Most importantly, the program 
needs management endorsement of a solid roadmap to improve ТАУ over the next year. 


In Section 6.0, Opportunities for Improvement, the Assessment team describes the barriers and 
provides recommendations to help FEMA move ТАУ to an optimum state. Although the team is 
recommending the Manhattan solution over a different COTS package (Oracle) or a custom 
solution (LIMS), it is imperative that FEMA understands and addresses its current issues with 
ТАУ if it is to move forward with a solid automated solution for its supply chain and asset 
visibility. FEMA cannot ignore these problems. expecting them to go away when it expands its 
supply chain to incorporate 3PL or 4PL providers. Instead, the integration of FEMA’s supply 
chain with other parties will only exacerbate the problems FEMA is experiencing today. 
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Section 6.0: Opportunities for Improvement 


This section of the Assessment describes the issues with the current state of TAV and provides 
recommendations to help FEMA move ТАУ to an optimum state. 


6.1 Information Technology Support 


The capabilities of the current TAV systems have been constrained by limitations in the FEMA 
IT environment. The effectiveness of the supply chain applications FEMA adopts in the near and 
long-term will depend on a robust IT operational environment. Some of the constraints, 
specifically those related to the Mount Weather hosting center and COOP, should be resolved 
over the next few years, as FEMA migrates its applications to DHS hosting centers. Others will 
require additional actions from FEMA IT. 


The Office of the Chief Information Officer (OCIO) is aware of these limitations and is working 
to address them — for all applications, not just TAV. The ТАУ PM has conducted some very 
promising meetings with FEMA IT during September 2007. There appears to be positive 
movement to address the issues in the future. However, FEMA IT is suffering from severe 
limitations on funding, and significant change is always a slow and complex process. 


The recommendations below describe specific areas of IT that have had an impact on TAV 
capabilities. If not addressed, these IT constraints will limit the effectiveness of the disaster 
supply chain no matter what applications solutions FEMA adopts. 


Poor network performance severely impacts the use of TAV 


The FEMA network environment presents challenges for the effective deployment of integrated 
supply chain system capabilities. These issues fall into general Wide Area Network (WAN) 
issues, and the "last mile" problem for disaster sites. In both cases, the result is an inability of the 
system to respond to users quickly enough to support their needs. The results are: 


* Users spending excessive time to get their work done; 
* User dissatisfaction and complaints with the system; and 


* Users do not use the system as intended and fall back on manual processes — which has 
downstream impacts since the data is not in the right systems when needed. 


Wide Area Network (WAN) 


The TAV TPM/WM environment has suffered complaints of slow network response from 
established field sites such as Camp Beauregard and the Fort Worth DC since it was implemented 
in FY2006. Since the Manhattan TPM and WM products are used successfully by overseas 
customers whose servers are in the US, it is clear that the applications can operate over an 
appropriate configured WAN. (Oracle similarly reports that it has overseas customers who 
operate from remote servers in the US and find response time adequate, even for warehouse scan 
gun response.) 


Since fall 2006, the ТАУ program has been working with the CIO's office to obtain a map of 
РЕМА wide area network to identify potential bottlenecks that might be affecting response time 
— e.g., multiple hops, saturated links, etc. As of the end of July 2007, the program was only able 
to assemble a map of the core network environment out to the Regions. The CIO's office is still 
working with the Regions to develop a network map of the Regional networks, including DCs 
and other sites in the regions, which would need access to TAV systems in a disaster. 
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Preliminary analysis of the core network environment identified several links, e.g., Boston, 
Chicago, Missouri, that are routinely running up to 100% capacity sometime during the day. 
Other core links such as from Mount Weather to Denton are running at over 50% utilization — all 
during a period without major disasters. 


The TAV WAN analysis also looked at the various protocols running over the WAN and 
determined that the secure web browser protocol (HTTPS) used by all the Logistics systems 
(except the eSuite) took up an average of only 4% of the bandwidth (1% to 8% depending on the 
link.) Thus, the FEMA Logistics systems did not appear to be a major contributor to the total 
WAN load. 


One sign of progress is anecdotal evidence that TPM Intranet latency significantly improved in 
late September 2007 when the TPM servers were moved behind the FEMA firewall for security 
reasons, The latency impact needs to be measured and confirmed under both normal and major 
disaster conditions. It also still leaves open the question of TPM latency from AirCards (see 
below) and the Internet (e.g., by vendors). 


An integrated end-to-end real time supply chain requires a robust WAN environment — both 
within FEMA and to its partners. WANs need to be carefully architected, implemented, and 
managed/tuned to be effective. Given the information the program has been able to acquire, and 
the difficulty obtaining timely WAN configuration and usage information, it is still unclear 
whether the FEMA WAN can effectively support disaster supply chain requirements. 


Last Mile Network 


The general WAN issues facing FEMA are similar to those facing any large, diversified business. 
However, FEMA’s need to provide connectivity to users supporting disasters in the field at 
transient sites such as FOSAs and PODs raises significant unique challenges. Modern supply 
chain systems rely on capturing data in real-time at the point of the event, This means that 
Logistics users need reliable access to the systems in the field from the start of a disaster. 


Traditionally, field access has been accomplished by equipping FEMA personnel with FEMA 
laptops with Verizon AirCards to establish a VPN connection into the FEMA Intranet. However, 
Logistics users and business sponsors have repeatedly stated that this approach to connectivity 
from the field is inadequate. Not all responders have laptops, and response time over the AirCard 
is often slow (ТАУ response time measurements showed that AirCard response was several times 
slower than landline response from DCs; sometimes slower than an old-fashioned dial-up 
connection). Again, the result is that users spend excessive time using the system to accomplish 
their work, become dissatisfied with the system, and in many cases bypass the system by relying 
on manual processes and bringing the system up to date when they can. 


Recommendations for addressing this problem include: 


=  Allowing users to access TPM from any connection and computer they can obtain access to -- 
ТАУ has been actively pursing permission from FEMA/DHS Security and CIO for non- 
FEMA computer/Internet access to TPM since FY 2006; 


* LIMS Program Office's suggestion of immediately deploying satellite connections to disaster 
sites and developing a low bandwidth version of the LIMS application specifically for that 
environment; 


* Working with Verizon to investigate and address slow AirCard network response; and 


* Having field users collect data manually and then call/fax/email it to Regional (еЕОЅА) or 
Headquarters (TPM, eTasker) staff for entry into the system. 
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No matter what specific applications are selected by Logistics, FEMA IT needs to address the 
issue of more robust connectivity from disaster sites to support an effective real-time disaster 
supply chain system. 


The hosting environment does not adequately support a mission critical system 


Mount Weather is FEMA's hosting site for the FEMA Logistics supply chain systems’. 
Unfortunately, the reliability of the IT infrastructure at Mount Weather does not appear to be 
adequate for a mission critical, real time environment such as the disaster supply chain. As shown 
in Table 15, the ТАУ server environments at Mount Weather have suffered several outages 
during the period of this assessment due to air conditioning, electrical, and infrastructure 
equipment failures. This has resulted in almost 14 hours downtime during June/July. This falls 
significantly below the OMB Federal-wide Infrastructure Optimization Initiative's availability 
targets of 2.4 hours per month downtime for even the lowest tier of Federal data centers (Table 
16.) 


Table 15 - Recent Infrastructure Outages at Mount Weather 


Prepare. Respond. Deliver. 


I TAV 
[ Date Problem Йрудйше. 
June/July Air Conditioning failure shut down Test and Development Lab (TDL) | Months (TDL) 
2007 servers 
6/21/2007 Power failure shut down production servers 3 hours 
7/6/2007 Router failure prevented user access to production servers 4 hours 
7/30/2007 Air Conditioning failure shut down production servers 6:45 hours 
8/25/2007 Air Conditioning failure shut down the ISAAC TDL servers, which in | Several hours. 
turn shut down ability to access TAV pre-production servers, used for | over the weekend 
production backup, testing, and training 


Table 16 - Federal Infrastructure Optimization Initiative Data Center Availability Targets 


I 99,671% 2.4 hrs 

п 99.741% 1.9 hrs 
ш 99.982% .12 hrs (8 minutes) 
IV 99.995 % -034 hrs (2 minutes) 


The level of hosting infrastructure downtime outlined in Table might have severe impact if the 
incidents occurred during a major disaster. The downtime also tends to decrease user’ confidence 
in the applications and encourages manual workarounds. 


The Mount Weather environment is also severely capacity-constrained in terms of space, electric 
and air conditioning. For example, a recent TAV request to add just three servers to the 
production environment and three to the pre-production environment is requiring a detailed 


3 eTasker has been hosted at FEMA headquarters at 500C Street with its backup servers at Mount 
Weather. However, based on direction from FEMA IT, the eTasker servers at Mount Weather will become 
the operational environment this fall. It is unclear if the current eTasker environment at FEMA 
headquarters will be retained as a backup. eTasker reports less infrastructure down time at the HQ site than 
TPM/WM report at Mount Weather. 
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analysis by Mount Weather operations to determine if sufficient space, electric power and air 
conditioning are available. 


FEMA IT reports that FEMA will move to DHS data centers in the FY2009-FY 2011 timeframe. 
During a meeting in late September 2007, the newly appointed TAV IT advocate mentioned the 
possibility of some FEMA applications moving to a DHS hosting site in first half calendar 2008. 
The TAY program needs to determine if this is a viable possibility for improving hosting support 
before hurricane season 2008. TAV also needs to work with IT to minimize hosting risks while 
ТАУ remains hosted at Mt Weather. 


TAV lacks a solid COOP/Disaster Recovery capability 


Except for eTasker, the Logistics disaster supply chain applications do not currently have 
COOP/disaster recovery sites to continue operations if the Mount Weather environment becomes 
unavailable. FEMA IT is developing plans for implementation of a COOP site for FEMA 
applications in general, but no specific information is available about when ТАУ will have a 
COOP site. 


Effective COOP/disaster recovery is not only mandated by Federal guidance, but an essential 
capability for an integrated, real-time, supply chain environment, providing critical disaster 
support to citizens. 

FEMA’s planned move to leverage DHS data centers in FY2009-FY 2011 (or even calendar 2008, 


see above) should mitigate this risk in the long term, but lack of COOP is a significant risk for the 
near term. 


System security decisions need to address business tradeoffs 


FEMA has traditionally enforced strict security controls on its systems; its inclusion in DHS has 
only increased the focus on security. Unfortunately, security constraints are generally presented to 
TAV and its business owners as absolutes and there has not been an effective mechanism for risk- 
based balancing of security and business needs. 


One major disconnect seems be over the "sensitivity" of the FEMA supply chain information. 
From the business owner perspective, the data appears to have low sensitivity and be widely 
available; in fact, some data such as shipping manifests for commercial carriers of FEMA assets 
are public by law. In contrast, IT Security has classified all FEMA data as at least Sensitive but 
Unclassified, triggering a number of FEMA and DHS security controls. Their view is that 
information on FEMA assets and shipments could be valuable to enemies trying to disrupt 
disaster relief or steal material, or embarrassing if released publicly. This wide variance in views 
needs to be resolved to support rational security/business tradeoffs going forward. 


There are also variations in the security constraints imposed on the various Logistics systems that 
have particularly had impacts on the current ТАУ Manhattan environment: 


* TPM/WM were required to integrate with ISAAC as a single sign-on front end while none of 
the other Logistics systems uses ISAAC*. The ISAAC integration took over a year and was 
only deployed in August 2007. It has also created a new single point of failure for the 
TPM/WM applications; when ISAAC is down for any reason, TPM/WM cannot be used. 
This has resulted in 33 hours of TPM/WM downtime in June/July due to ISAAC downtime. 
It has also made it impossible to use the ТАУ Test and Development Lab (TDL) environment 


+ LIMS does not employ the same level of ISAAC integration as TAV; it makes a simple call to 
ISAAC when LIMS passwords are created to validate that the password meets FEMA strength rules, 
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at Mount Weather since there is currently no ISAAC support for that environment, and testers 
cannot log into WM/TPM without ISAAC. There is no ISAAC backup production? or COOP 
site, which further increases the risk of ISAAC as a single point of failure for multiple TAV 
systems. 


* Because of restrictions on "contractor" access to production systems, Manhattan technical 
support staff are not granted even limited read access to the TPM/WM production 
environment to help in troubleshooting. In contrast, the eSuite is administered by its 
development contractor, and LIMS is operated primarily by Verizon — whose subcontractors 
developed and maintain the application. 


* Similarly, the TAV program has requested Manhattan contractor access to the TDL 
environment to upload new patches for over a year. This issue has still not been resolved with 
FEMA security. 

* TAV has been unsuccessfully pursuing Internet access to ТРМ by FEMA employees and 
partners since mid-2006. The only current Security permission for external access is by 
vendors (of which, there are currently none.) Therefore, ironically, FEMA employees at 
disaster sites with connectivity issues cannot use the Internet to access TPM to get their jobs 
done, while FEMA's Security Group has allowed Internet access by vendors. 

* DHS has suggested providing TPM access through a CITRIX intermediary, but they are still 
establishing a robust CITRIX environment and it has not been tested with TPM or other 
FEMA mission critical systems. In contrast, IRRIS, which has FEMA and DLA shipment 
location and content information, is available on the Internet to anybody with an IRRIS ID 
and Password. 

= TPM/WM are required to use HTTPS for Intranet access, despite concerns about its pos 
contribution to the network latency problem, while the eSuite applications use HTTP for 
Intranet users. 


ible 


Flexible access and integration amongst all the participants (FEMA, OFAs, states, vendors, etc.), 
and system access from the field to record events where and when they happen are core 
foundations for a modern, integrated supply chain system. Whatever software applications FEMA 
selects, a more flexible security infrastructure and an effective process for flexibly balancing 
security risks and business mission needs will be critical for success. 


Lack of integration and data warehousing infrastructure impedes system usability 


None of the current Logistics systems connects to an effective FEMA or DHS integration or data 
warehouse/business intelligence infrastructure. These capabilities are critical to a modern, 
integrated supply chain system that depends on real-time integration among multiple partners and 
systems, and “intelligent” use of information across end-to-end processes. 


DHS has established an Enterprise Service Bus infrastructure to support its move toward a SOA. 
However, there does not appear to be any comparable infrastructure within FEMA that would 
allow both FEMA systems* and partners to “plug into” a modern integration framework for real- 
time sharing of supply chain events. 


Both DHS and FEMA have established data warehouse initiatives for a standardized 
infrastructure for combining, analyzing, and reporting information. However, the FEMA data 


5 The ISAAC team is currently investigating the possibility of a backup production environment. 
5 The Manhattan systems include an integration layer, which can plug into a wide variety of ERP 
integration frameworks (e.g., Oracle, SAP), and integration products (e.g., WebMethods). 
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warehouse does not appear able to input data from non-Oracle databases, e.g., the SQL Server 
used by TPM and eSuite. Even Logistics systems such as WM and LIMS that use the Oracle 
database management system do not currently share data with the data warehouse. Thus, each 
FEMA Logistics system currently relies on its own separate database for reporting’. This leaves 
each system's information cut off from both the other Logistics systems and related systems such 
as Finance or Acquisition. 


No matter which applications Logistics selects, an effective integration and data warehouse 
infrastructure will be key to successfully implementing and operating the disaster supply chain. 


TAV IT support is fragmented and lacks accountability 


TAV is comprised of operational information systems that support a mission critical function. 
Although the Logistics Directorate owns and manages the systems, TAV requires support from 
FEMA's IT Department. Any successful IT system must have shared responsibility and 
accountability from both the business unit that owns the system and the IT department that 
supports it. To date, the IT support for TAV has not provided the focus and responsiveness 
required to sustain a system of TAV's size, complexity, and mission. 


The current mechanism for acquiring IT support for TAV can come from one of three primary 
paths: 


* Help Desk Escalation- When real-time system issues arise, affected users call into FEMA's 
Help Desk where issues are escalated to IT through a three-tiered escalation process. 


= Operations - There are IT staff whose job is to manage the infrastructure that runs TAV, 
manage ongoing security, etc. 

* Other - For all other IT requests, the TAV group has been working through a liaison to 
channel IT issues, requests, and support. Examples of these requests include addressing 
network response time, the need for more servers, reviews of technical documentation, data 
archive strategy, etc. These requests can also include questions or concerns with the first two 
paths, escalated Help Desk issues, or problems with Operations. 

One of the bigger issues facing TAV’s communication with IT is that there has not been a 

focused contact person or group accountable to support TAV. No one person or group takes a 

holistic view of TAV. The TAV program funded an IT liaison position to channel TAV requests 

to IT. Although a good concept, this role has not proven effective. Although the TAV program 
funded the liaison position, it is staffed by IT with a contractor resource who reports to the IT 
project management group. From May 2007 (when the PMO established the formal Integrated 

Project Team (IPT) process) through August 2007, the liaison consistently reported that obtaining 

responses from IT personnel was problematic and unreliable. The liaison had no authority to 

manage the issues or to escalate problems directly to IT management, except through the IT 
project management group. This structure has resulted in layers of people between the between 
the project and the actual resources that can help resolve the IT issues, resulting in excessive 
delays. 


The TAV IPT has been tracking issues since April of 2007. The following examples illustrate the 
support issues: 


? Manhattan provides a separate application — Performance Management — for data warehouse/business 
intelligence support for its applications. FEMA has just purchased the PM application, but has not (yet) 
implemented it. 
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* Network Latency — TAV users reported very slow response times in the field. The TAV IPT 
formally escalated the issue to the IT liaison on 5/10/07. As of 10/09/07, the issue still 
remains “open” and unresolved. 


* Data Archiving — To improve the user experience, and to mitigate response time issues, the 
ТАУ team requested that 2005 and 2006 transaction data be archived. The issue was formally 
escalated to the ТАУ IT liaison on 5/22/07. As of 10/09/07, the issue remains “open”. IT is 
still identifying what server to use for archiving. 


The ТАУ PM met with the IT team at Mount Weather to conduct an early read-out of this 
Assessment and to raise the issue of support on September 12, 2007. At that meeting, a senior 
federal employee from IT was identified to the TAV program as the IT Advocate. This is very 
positive, as this role appears to provide TAV with a single point of contact for IT issues. This 
role, however, is in addition to the IT liaison role, which is not clearly defined. In addition, the 
leader of the Operations group stepped up and stated that he was the point of contact for issues 
that go through the Help Desk and require IT resolution; however, he also noted that he does not 
have enough funds or staff for full support of TAV. The groups agreed to continue working 
through the issues. One area of concern was that most of the IT group at Mount Weather was not 
aware that there was an IT liaison for TAV. This is very troubling, because TAV was directed to 
work all IT issues through the liaison. 


The Assessment team believes that the IT issues are on a path to resolution; however, the team 
recommends the following to ensure that the relationship remains productive: 


* ТАУ PM and leaders from the IT project management group and user liaison group meet and 
define the role of the TAV Advocate. What are the responsibilities for the role? In what areas 
of the program will the advocate be involved? 


* ТАУ PM and leaders from IT project management group and user liaison group meet to 
determine the role of the ТАУ liaison; this is a role that is funded by ТАУ and to date has 
been ineffective in raising and resolving issues. 

«IT and the ТАУ PMO meet and define the program's relationship with the Operations 
Manager. Does the PM need to work through the liaison or advocate to contact the 
Operations Manager? What are the specific responsibilities for the role? When and how will 
the Operations Manager be involved with TA V? What are the Operations Manager's issues 
with funding — and what amount of support can this role provide to TAV? 


* The TAV PM should require that someone from IT step up to define the IT roles critical to 
TAV and to ensure that each IT representative understands his/her role and the other roles, 
with respect to TAV. In addition, IT should provide a service level agreement (SLA) defining 
the level of service that will be provided to TAV. 

* Logistics and IT senior management should track the IT issues via a dashboard for weekly 
review until they are fully resolved and all outstanding issues are on track. The activities 
should not be considered under control until all of critical IT roles have been defined in 
writing, and all of the current TAV IT issues have an owner in IT and an associated project 
plan. 
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6.2 Current TAV Implementation 


The Assessment team conducted interviews with TAV systems owners and users and analyzed 
the current processes supporting TAV. The team observed many issues with the current system 
implementation that are preventing FEMA from getting the full value from TAV. This section 
describes several issues with the current implementation -- problems with the development 
methodology, lack of integration, inefficient system administration, and the high costs associated 
with TAV. 


TAV is impacted by a lack of standard IT architecture and development 
methodology 


One of the major causes of problems with the current TAV implementation is that FEMA has not 
employed standard industry/Federal IT architecture and development best practices to TAV. The 
TAV solution architecture was not designed from the top down, based on solid business 
objectives. The TAV solution consists of several software applications selected and developed 
separately to support individual business needs. It was only after the fact that they were patched 
together as an overall TAV solution. The primary components of TAV consist of: 


* Manhattan - Selected in an “urgent and compelling" way because of issues with FEMA's 
response to hurricanes in 2004. FEMA senior management selected the integrator in a sole 
source acquisition and allowed the integrator to select the Manhattan package. FEMA 
required the integrator to configure the system very quickly, based on requirements from the 
Property Management group. FEMA invested heavily in warehouse management software 
and never integrated it into its business processes. As a result, the user community has never 
really accepted the software and FEMA pays the integrator to execute many of the functions 
that FEMA staff should be doing. 


* eTasker - evolved out of a need to automate spreadsheets to support FEMA's Transportation 
function. It was expanded to automate FEMA business processes in the LMC. Although the 
design and implementation for eTasker occurred at the same time as the TAV implementation 
and affected the same user population, FEMA Logistics did not ensure that integrated and 
unambiguous business processes supported the projects. Nor did FEMA IT or EA identify 
and prevent the overlap of functionality and the duplication of data. eTasker covers many of 
the functions and data that the Manhattan system does and is seen as duplicate to Manhattan 
by the user community. 


This method of building point solutions to solve specific problems creates data duplication, 
excess costs, lack of user acceptance, and in the end, FEMA is not getting the information it 
needs to manage its supply chain and gain asset visibility. The information is scattered between 
multiple systems and manually maintained spreadsheets. 


These problems continue to this day. The eTasker contractor worked directly with field 
representatives to design SPOTS and eFOSA for very specific field requirements. The solutions 
are not integrated amongst themselves, nor with eTasker or with TPM. Even though the goal of 
TAV is a central repository for all requests for disaster supplies, the eSuite creates separate 
repositories for requests depending on the business function and its location. In addition, there are 
clear overlaps with the Manhattan implementation. SPOTS and eFOSA have not rolled out at the 
time of this Assessment, but they appear to be on the path to implementation. Although the field 
staff will benefit from the automation provided by the applications, FEMA can also expect issues 
resulting from lack of integration and duplication of user business functions. 
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Most IT systems evolve over time to achieve a clear set of business objectives. There is no road 
map today for TAV to achieve its business objectives. Listed below are two problems that could 
be resolved over time, but today are not on a roadmap for resolution: 


= As part of the Assessment, the team developed a visibility matrix (Appendix D) to assess the 
level of visibility for FEMA supply chain transactions. The analysis shows FEMA only has 
in-transit visibility for the subset of the shipments that originate from the Fort Worth and 
Atlanta DC and are transported by FEMA owned or leased trailers. The agency has very 
limited visibility of vendor, OFA shipments, shipments outside of Regions 4 and 6, and those 
transported by commercial carriers. 


* Today, FEMA does not have a consolidated view of requests, orders, shipments, and 
inventory of commodities to support disasters. The information is scattered between multiple 
systems. Most importantly, there is no blue print to show how, over time, the TAV system 
will evolve to resolve these issues and improve asset visibility. 


The solution to these problems is a solid architectural blue print for TAV, and a road map to 
improve functionality and reduce duplication over time, managed by standard IT development 
practices. The problems outlined in this section have nothing to do with the software packages — 
they are the result of a lack of standard IT management. The Assessment team recommends the 
following actions to improve the current TAV implementation: 


* EA and the ТАУ program office should develop а solution architecture for ТАУ that defines 
the major functions and the system components that will perform each function. FEMA 
Logistics management and IT management should approve this architecture and enforce 
compliance with it. This solution architecture should include a “to be" state, defining future 
interfaces and system modifications to improve asset visibility and user acceptance; 

* FEMA should develop a road map and release schedule for ТАУ that implements the "to be" 
TAV state over time; 


* The ТАУ PM must approve all requests for new functionality for any of the TAV systems 
before the functionality is built; 


* Development teams must create requirements documents for all new functionality. The TAV 
PM must sign off on the documentation. The TAV PM will enforce that all TAV stakeholders 
are included in the requirements; 


* Development teams must create comprehensive project plans for development activities, 
which address the roles of all impacted stakeholders, including IT and the ASB; 


= For major enhancements, the technical teams must develop design documentation and 
conduct technical design reviews with IT, EA, and all of the TAV development teams; 


* FEMA should establish a TAV users group, led by the TAV program office, to track ongoing 
user issues and enhancement requests. The group should include representatives from 
business functions impacted by TAV and representatives from each of the TAV development 
teams; and 

* FEMA should conduct an in-depth design review of the SPOTS and eFOSA systems before 
they are rolled out. The review should include FEMA Logistics representatives, FEMA IT 
and EA representatives, and the TAV development teams. In addition, the overlap between 
the Manhattan solution and the SPOTS and eFOSA solutions should be resolved and a clear 
set of business processes established before the applications are rolled out. 
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Lack of integration and data sharing among existing systems limits effectiveness 


The lack of integration and data sharing among the existing systems limits FEMA’s ability to 
leverage the capabilities provided by the individual systems. In terms of support for an integrated 
end-to-end disaster supply chain, “the whole is less than the sum of the parts.” 


Figure 2 below shows the current Logistics systems, as well as other FEMA systems and partner 
systems that are part of the current overall, end-to-end disaster supply chain. The figure shows the 
existing interfaces among the systems in blue dashed lines*. As can be seen from the scarcity of 
blue lines, most of the FEMA Logistics systems are currently islands separated from each other, 
the rest of FEMA, and FEMA’s partners. 


The red solid lines in Figure 2 show potential interfaces across the supply chain. This includes 
interfaces among the FEMA Logistics systems — particularly between eTasker (request) and TPM 
(orders), and between WM (warehouse management) and LIMS (property management). The 
figure also highlights potential interfaces between FEMA’s partners and FEMA Logistics — in 
particular interfaces from vendors (and other “suppliers”) to TPM. The high priority interfaces 
are circled. These include: a) an interface between eTasker and TPM to push requests to TPM; 
b) an interface between WM and LIMS to address duplication in inventory tracking; and с) an 
interface between TPM and eFOSA to push FOSA requests to TPM. Finally, the figure shows 
potential interfaces from the Logistics systems to other FEMA systems such as Acquisition and 
the data warehouse needed to tie Logistics into the overall FEMA environment. 


® There may be additional existing interfaces among the partner systems or other FEMA systems that 
do not involve the FEMA Logistics systems. 
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Figure 2 - Existing versus Potential Integration of Disaster Supply Chain Systems 
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The number of potential interfaces in Figure 2 appears daunting. To some extent this reflects the 
inherent interconnected nature of an integrated supply chain. However, some of the complexity is 
due to the number of overlapping systems in the current Logistics environment — e.g., SPOTS, 
TIMACS, and eTasker all support requests. The apparent complexity also reflects the current 
environment where each interface is developed as a point-to-point solution, rather than leveraging 
a common integration infrastructure (see Section 6.1). Figure 3 depicts a notional high-level 
integration diagram with Logistics, other FEMA, and partner systems all "plugging" into a 
common integration infrastructure to support visibility across the end-to-end disaster supply 
chain. 
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Figure 3 - Potential Integration Across the Disaster Supply Chain 
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Besides the lack of interfaces, there are also data standardization issues limiting the effectiveness 
of the current systems environment. 


Table 17 shows a CRUD matrix (Create, Read, Update, Delete — not used) developed in support 
of the TAV EA. The CRUD matrix is used to identify commonality of data usage across systems. 
Typically, in a well-integrated environment, most data will be created in one system and then 
read (or perhaps updated) in multiple systems. 


Table 17 shows, while many of the current systems use the same types of data, most of them 
create data for their own use, rather than sharing data among the systems. This not only results in 
duplicate functionality and effort, but also makes data sharing difficult, since each system is 
likely to create similar data using its own proprietary format and business rules. For example, two 
of the most important data elements for tracking requests, orders, shipments, and inventory across 
a supply chain are the "Item" and the "Site" (shipping and receiving.) Unfortunately, many of the 
systems (including TPM/WM, LIMS, eTasker, eFOSA, SPOTS, TIMACS, and Acquisition) have 
their own names and structure for their individual master Item? and master Site lists. This makes 
it difficult to integrate systems or provide aggregate reports on activities across systems. 


? There is a move in Logistics towards using National Stock Numbers for items, which may make it 
easier to match to the TPM/WM master item list. 
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Integration within FEMA and with its partners is critical for an effective disaster supply chain. 
The lack of integration and the duplication of data within the current environment must be 
addressed no matter what systems are selected as the near and long-term foundation of TAV. 


System administration functions must be optimized to support TAV 
source of problems and confusion for TAV users. Several 


System administration of ТАУ i: 

factors contribute to the frustration: 

* Each system of the TAV solution is administered by a different group — new users to TAV 
must work with three different groups and three different processes to gain access to the TAV 
systems. 

* Тһе еТаѕкег development team also administers the eTasker application. They manage 
users and other system data. 

* IRRIS is administered by the GeoDecisions, the IRRIS development contractor. 

* Manhattan, on the other hand, is administered by the ASB (a component of the Logistics 
Property Management Division), whose primary role is to support and administer the LIMS 
system. 


* The ASB staffing model and business processes were designed for optimal support of LIMS 
property management and not TAV supply chain management. This has resulted in several 
challenges for TAV users. 

*  Theentry of locations by the ASB after hours has been a problem for TAV users. As 
disasters unfold and FEMA determines the locations for FOSAs, some of the FOSA 
locations may not exist in TAV. Since Commodity Managers must select the location as the 
destination for orders, they must contact the ASB when the location is not already in TAV. 
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When this situation occurs outside of normal business hours, it has taken several hours for 
the ASB to create the location, consequently delaying the order. 


* Тһе same situation occurs with the entry of new users. Every time a user is required to 
work at a different location, they must contact the ASB to update the access in the TAV 
system. During disasters, if this service is required outside of business hours or on 
weekends, users can wait several hours to get access. 


* Also, the entry of vendors into ТАУ has been reported as a problem. An example provided 
to the team by one Commodity Manager is that he had to wait for over a week to add a 
vendor to TPM. Obviously, the shipment could not wait until the vendor data was entered 
so the Commodity Manager had no choice but to find a work around, invalidating the TPM 
data. 


* In addition, because the ASB staff primarily support LIMS and reside at Mount Weather, 
removed from the TAV team, they do not appear to have fully embraced the system. 
Typically, system administrators are super-users who deeply understand the nuances of the 
system. The ASB staff continues to rely on the Manhattan staff to answer questions and 
resolve issues. 

* One complicating factor is that ASB does not have a way of paying its staff for after hours 
support until a ster has been declared. This limits their ability to respond 24x7 during 
the initial periods leading up to the disaster declaration. 


The Assessment team recommends that FEMA determine a standard method of system 

across all of the TAV systems. Most importantly, FEMA should develop a set of 
ation processes based on the real time, mission critical operational needs and 

f the TAV environment. The administration processes and policies should be applied 
equally to all of the TAV system components. For example, when TAV team members have 
proposed that the TAV Resource Accountability Coordination Center (TRACC) be utilized for 
system administration, the ASB has stated that only federal employees can administer systems. If 
that is the case, then why do contractors administer eTasker and IRRIS? 


If the ASB continues to administer TAV, then the TAV PM and ASB manager should agree on 
operating practices and develop an SLA outlining a clear understanding of services provided. They 
should also consider broadening the scope of administration to eTasker and IRRIS as well. 
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ASB support for TAV Help Desk and Testing requires improvement 


In addition to system administration, the ASB provides two critical roles for TAV — they staff the 
ТАУ Help Desk and conduct independent testing of the system. There are problems in both areas 
that require attention, and if improved, could positively affect TAV. 


* ТАУ Help Desk — Although the ASB has been supporting the TAV Help Desk since the 
systems’ implementation, they have only been entering trouble tickets into FEMA’s official 
Remedy database since mid-September 2007. When the TAV team met with IT in mid- 
September 2007, they learned that the only way to get IT attention for system problems is via 
Remedy tickets. Prior to that meeting, the ASB was not regularly entering Remedy tickets for 
user-reported ТАУ issues. In addition, the ТАУ PMO does not receive standard reports that are 
typical of Help Desk operations, e.g., number of calls per time period or number of open issues, 
nor has the escalation path for trouble tickets ever been explained to the TAV team. Finally, the 
Assessment team received reports that, especially after hours, the ASB staff that respond to 
Help Desk calls do not understand the applications adequately enough to address basic user 
issues. 


The Help Desk functions for TAV today are fragmented, as the ASB does not handle calls for 
eTasker and IRRIS — help for those systems is provided by their respective teams. This creates 
issues for users, who must remember multiple processes for getting help for the systems and 
furthers the perception of separate systems instead of one TAV solution. In addition, it creates a 
situation where the TAV PM must go to multiple teams to get a picture of system and user 
issues affecting TAV. 


Assuming that the ASB remains responsible for the TAV Help Desk function, the Assessment 
team recommends the following improvements: 


* Development of a Help Desk manual for ASB staff, listing the typical problems users 
encounter — This type of job aide would ensure consistency in answers to users and 
improve the results when less experienced ASB staff man the Help Desk; 


* Formal documentation of the Help Desk process by the ASB ~ the TAV team needs to 
understand Help Desk processes and the escalation path for issues; 


* Generation of standard reports from Remedy, provided weekly to ће ТАУ PMO, and 
reviewed weekly at the TAV IPT meeting — This would include the ASB submitting the 
weekly reports to the IPT lead each week. An ASB representative would have an agenda 
item on the IPT to summarize the number and types of tickets submitted that week and a 
review of the remaining open issues; and 


* Consider expanding the Help Desk to include IRRIS and eTasker — this would help users to 
see TAV as an integrated solution. It would also facilitate the support of TA V, in that all 
issues would be subject to a common process. 


* ТАУ Independent Testing — Although FEMA has an IT Quality Assurance (QA) group that 
performs independent testing, decisions early on in the life of TAV resulted in the ASB 
conducting testing for TAV. The ASB conducts their tests after the technical team has 
completed their development and unit testing. They take the technical team scripts and execute 
them. This type of testing only tests that the technical team's test scripts work. Standard 
industry practice is that an independent testing group develops its own test plans and scripts to 
ensure testing beyond the perspective of the technical team. 
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From the ASB perspective, they report that they do get requirements documents up front to 
assist with testing preparation and that there is typically no release schedule or project plan 
from the ТАУ team to help them plan for testing. 


Finally, eTasker and IRRIS are not tested today by the ASB. The eTasker team conducts the 
system testing. They do, however, include users in their testing, but there is no independent test 
group beyond the users. IRRIS testing methodology is unknown, 


Assuming that the ASB remains responsible for ТАУ testing, the Assessment team 
recommends the following improvements: 


« Development of testing processes for TAV releases - ТАУ PMO, ASB, and FEMA 
representatives should meet, determine, and document a standard process for the 
independent testing of TAV. The process should initially take into consideration the current 
constraints on ASB and QA staff; 


* Improve project management processes to ensure all testing is planned up front ~ ће TAV 
team should document requirements for all releases and include the ASB in the review 
process. In addition, they should develop comprehensive project plans that include testing. 
ASB staff should review and contribute to the project plans to ensure their buy-in and 
commitment to the schedule; and 


* Consider expansion of testing to include IRRIS and eTasker — this will be increasingly 
important as the applications become more integrated. It will ensure testing of a full end-to- 
end process — irrespective of whether the applications are technically integrated. This type 
of integrated testing will ensure consistency across the applications. 


FEMA should consider modifying its current integrator strategy to reduce 
contractor costs and improve service levels 


The Assessment team recommends that FEMA work with Acquisitions to re-evaluate several 


areas of the current strategy to improve service and reduce costs. 


* FEMA is paying a high monthly premium for the ability to surge to a "rolling 100"? 
Siratix/Manhattan staff to support disasters. This level of support would only be used for a 
major disaster. This insurance covers staff to enter data into the Manhattan system and to 
perform transponder operational tasks. 


The Assessment team recommends that FEMA consider other options to ensure data entry 
during a disaster and reduce its monthly premium to Stratix. 


* One option that the current TAV PM has researched is to leverage the DAE process to train 
a pool of DAEs to support TAV. The DAEs could also be cross-trained to perform 
additional functions, e.g., eTasker. This would reduce costs by leveraging processes 
currently in place in FEMA. 

* Another possibility might be to utilize the current APO staff to cover ТАУ functions. This 
might be a realistic option if the consumable commodities are eliminated from the property 
management tracking duties of the APOs. 


* Finally, FEMA should consider developing processes to use the TRACC or the LMC as a 
backup during an event. FEMA could develop processes for calling or emailing the 


“Rolling 100" means that once any number of staff are deployed, Stratix commits to have an equal 
number ready to deploy; i.e., if 100 staff are deployed, Stratix has committed to having another 100 ready 
to go. 
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TRACC or the LMC to enter orders during disasters. FEMA could gain great efficiencies 
by having one group at Headquarters covering some of the extra burden during a disaster. 


* FEMA is paying for contractors to operate the Manhattan system. The integrator maintains 
several staff at each of the DCs and at Mount Weather to operate and administer the system. In 
addition, the integrator maintains a team to manage the transponders. In typical information 
systems implementations, agency staff are trained to operate the system and execute the 
required business processes, There are also, typically, super-users in each business location 
(e.g., each DC) who mentor other users and maintain the data integrity of the system. This 
approach has not been implemented for TAV, with super-user rights limited to the ASB. 


As with the discussion of the surge premium above, the Assessment team recommends FEMA 
focus on training its organic staff to operate the ТАУ system in the field and at Headquarters. 


* Тһе team recommends that FEMA reconsider this strategy and actively work to transition 
the operations of the system to FEMA staff. The amount of contractor staff supporting the 
system is excessive in comparison to the amount of order and shipment activity FEMA 
executes in a typical week. The transition will require support and enforcement by FEMA 
management, In addition, it will require training (already covered by the TAV contract.) 


* However, if FEMA wants users to accept and use the system, it must address the other 
major issues raised in this Assessment — IT network and support improvements, strong 
business processes, and a TAV release roadmap to improve functionality, decrease 
duplication and improve usability. 


" FEMA is paying high travel costs for out of town contractor employees. Most of the FEMA 
Stratix/Manhattan team does not reside in the place they work. As a result, FEMA is paying for 
airfare, lodging, and food for most of the team. Some of the roles require travel to multiple 
locations, but the bulk of the roles could be filled locally. 


* For example, FEMA pays for a revolving pool of TRACC staff to travel to Washington 
each week. FEMA would benefit from a consistent team — the same people at FEMA each 
week — who are local to Washington D.C. That strategy would improve the level of service 
in the TRACC and save money for FEMA. 


* In addition, there are several project management roles currently filled by Stratix staff that 
travel to FEMA each week. If FEMA decides to execute additional options with Stratix, 
then FEMA should ask that Stratix transition to local staff to reduce travel costs. 


"= The current integrator is not providing required systems integration support such as release 
management, IT project management, system interface development, or requirements 
development. FEMA should consider re-evaluating its current contract to specify support for 
these high value services, which would be expected in a typical software integration contract. 
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6.3 Property Management vs. Supply Chain 

This section addresses the issues FEMA is experiencing between supply chain management and 
property management. LIMS is FEMA's property/asset management system today. NOTE: 
although FEMA uses the term "property management," the Assessment team prefers the term 
"asset management," as the former typically can also include buildings and land. The team 
observed that FEMA is mixing together the philosophies of asset and supply chain management 
and recommends that FEMA de-conflict them among its IT systems and operating practices. 


FEMA must clarify and optimize the functions of property management and supply 
chain management 


FEMA needs to clarify the difference between property/asset management and supply chain 

management. The terms are defined below: 

* Asset Management - the day-to-day accounting and tracking of significant, long-term assets 
of an organization. Asset management includes: 

* Management of inspection and maintenance cycles; 

* Depreciation categorization (capitalized versus expensed) and other considerations leading 
to a clean financial sheet for audit; and 

* Changes to location, ownership, responsible parties, etc. 

Asset management should begin when an asset is either requested or purchased (entering the 

system) through its disposal (loss, consumed, destruction, sale, etc). 

* Supply Chain Management - the management of consumable material destined for customer 

(disaster victim) consumption. Supply chain management includes: 

* Providing demand forecasts to suppliers; 

= Ordering from suppliers; 

* Storing and processing material within warehouses or other processing facilities; 

* Transporting material inbound from vendors or outbound to customers; 

* Receiving and fulfilling requests from customers; and where feasible, 

* Providing in-transit visibility to supply chain managers and executive decision makers. 

There are many subcomponents to supply chain management, each with a specific meaning 

and purpose as well: 

* Supply Chain Planning — Long-term capacity and demand planning (3-12 months) based on 
demand forecasts. Demand forecast can be based on previous performance, trends, 
estimates, etc, The more accurate the forecast the better the plan. This may not be easily 
applicable to FEMA needs, as these are typically steady-state operating forecasts rather 


than Day 0 – Day X emergency response forecasts. However, FEMA should be able to use 
scenarios to develop demand plans in reaction to different disasters. 


* Warehouse Management — The specific daily operations of a warehouse facility to manage 
the material received, stored, and shipped for disaster relief. 


* Yard Management — The management of external parking and staging facilities, to 
prioritize truckload processing. This is often in conjunction with an adjacent or downstream 
process, such as a warehouse calling forward the next truck for loading, etc. 
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«Transportation Management — The ordering, scheduling, and execution of material 
transport from its current location to a specified destination. This may cover various modes 
(air, truck, train, sea) and involve optimization based on origin-destination pairs and most 
efficient routes and schedules based on an overall network of material moves. 


Both property management and supply chain management must have a predetermined method for 
ordering items, or order management. They must also have efficient ways to tie into the financial 
management side of the organization to reconcile receipts and keep clean auditable records, 


While the pure definitions of asset management and supply chain management may seem similar, 
they should not be confused because they are quite different in terms of philosophy and their 
measures of “success.” 
* The goals of asset management: 

* To account for all critical assets throughout their lifecycle for operational and financial 

purposes; 

* To facilitate intelligent, reasoned repair versus replace decisions; and 

* То provide vendor and manufacturer history to augment new acquisition decisions. 
* The goals of supply chain management: 


* To manage material as efficiently as possible in order to deliver the right material when 
needed to the correct location; and 


* То streamline processes to minimize bottlenecks and other restrictions that inhibits material 
flow. 
The philosophy of supply chain management is smooth, efficient material flow to final 
consumption, while the philosophy of t management is complete accountability of all assets 
wherever they are. While these philosophi. ould ideally be complementary, their goals are 
different, and, if taken to the extreme, will be achieved at the cost and performance of the other. 


The Assessment recommends that FEMA develop clear business rules that differentiate supply 
chain management versus accountable property management. Accountable property should focus 
on critical and high-dollar assets identified in official guidance, not by priority identified by 
individual APOs. FEMA should establish a definition of property management to include what 
type of items it will track, the dollar thresholds, maintenance priorities, and financial guidelines 
(depreciation, etc.). FEMA should use LIMS for tracking only those items that fit within the 
scope of the definition. 


There is a high level of duplication between LIMS and TAV 


The Assessment team observed that FEMA is tracking items of value that are normally below the 
threshold of accountable property practices in LIMS. LIMS is tracking everything in its 
warehouses from IT assets to cotton swabs, tarps, and water. Today APOs are using LIMS to 
receive, inventory, and transfer/ship consumable material such as water. The user community 
sees duplicate practices for receiving and shipping in TAV and LIMS. Most commodities are 
tracked in both systems with staff performing the same function, but using different systems — 
sometimes the same staff supporting both systems. 


Tracking IT assets, furniture, and other reusable high value items fits the goal of accepted 
property management practices. Consumables such as water and low value items such as folding 
chairs should be treated as supply chain inventory and left out of the property management 
system. Clear lines of distinction and business rules between property management and supply 
chain management would help to eliminate most of the duplication between LIMS and TAV. 
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FEMA needs to rely on its supply chain management system, ТАУ, to effectively manage its 
consumables and operate its DCs. The supply chain management solution should integrate with 
other FEMA systems, including the accountable property system for assets that must reside in 
both systems. 


FEMA’s accountable property management practices are affecting the overall cost 
and performance of its supply chain 


The Assessment team observed several instances where the stringent rules for property 
management were being applied to supply chain management operations, limiting the efficiency 
of the supply chain. This may reflect the reality that many of the staff interacting with TAV at the 
DCs are actually APOs or IMS's whose core experience and training is on the property 
management side of FEMA. 


The team observed that APOs continue to rely on paper documentation for shipping and receiving 
authorization. Executing time-sensitive actions, especially on life-saving and life sustaining 
consumable items should not be dependent on the execution of the property management 
functions, especially if based on a paper process like FEMA's PTR. 


Interviews revealed APOs conduct 100% inspections of “property” before creating/updating 
records in LIMS. This doesn't sound unusual if an item is by definition accountable property like 
laptop computers and satellite phones. However, if it's an inbound vendor shipment, contracts 
should dictate item and packaging quality, FEMA should not need to perform one hundred 
percent inspection. Random inspections should be the norm. 
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6.4 Business Processes 


Based on interviews conducted during the Discovery phase, the Assessment team documented a 
comprehensive set of process flows that reflect the existing work functions that support FEMA's 
supply chain. The process flows incorporate all of the ТАУ systems — TPM, WM, eTasker, and 
IRRIS — in addition to directly related LIMS processes. The high-level process flows are attached 
in Appendix В of this document. The team found that although ТАУ is in production, the overall 
supply chain processes still consist primarily of phone calls, emails, and entering data in 
spreadsheets or on white boards, with the systems simply utilized as an audit trail. There are 
many reasons why the systems are not used effectively, including poor response time and 
duplicate data entry; those issues are addressed in the other parts of Section 6.0. This section, 
however, addresses one of the biggest issues affecting use of TAV ~ the lack of established, 
measured and monitored business processes. 


FEMA Logistics lacks a set of managed and measured business processes, 
integrated between Headquarters and the field, and aligned to a set of overarching 
business objectives 


The primary observation made by the Assessment team is that existing processes focus on 
organizing individual tasks and/or functions, but do not work together to support a set of 
overarching business objectives. Current processes support one part of the supply chain or one 
system, but are not established with an understanding of how each process impacts and influences 
other parts of the supply chain, While the evolution of piecemeal processes and solutions has 
enabled the field to gain marginal efficiencies at the execution level, the lack of management and 
standardization has created a barrier to real-time situational awareness of the overall disaster 
supply chain, 


Processes are communicated by word of mouth or they are found in department-specific job 
aides, They lack formality and accountability. Even though the ТАУ solution is 
primary method of information transfer continues to be email, phone and spreadsheets. Many 
people at FEMA have worked together for years and know the shorthand. They have become 
skilled at the use of manual processes. Informal processes may work in times of minor events, 
when experienced personnel work together, but they cannot sustain the level of information 
sharing required to support a major disaster. Assessment interviews revealed that informal 
processes contributed to the FEMA’s problems supporting Katrina. The processes and systems 
worked when experienced personnel were executing them, but when the situation escalated and 
inexperienced personnel were included they failed. Although the systems and processes were 
both inadequate during Katrina, only the system issues were addressed with TAV — the process 
issues were essentially ignored. 


Today, separate development teams implement each of the TAV systems (Manhattan, eSuite, and 
GeoDecisions) and LIMS independently. Although the systems must work together to support the 
business functions, no one group addresses the overall business process. The systems are rolled 
out with a set of procedures that explain how to operate the system — and the users are left to 
develop new business processes to incorporate the systems into their work streams. 


The Assessment team recommends that FEMA develop and communicate a comprehensive set of 
business processes to facilitate the disaster supply chain, and the systems that support it. This 
recommendation is further defined below: 
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* The first step for development of the processes is to outline the strategic Logistics objectives 
that both Headquarters and the field require from the information systems supporting the 
disaster supply chain. The processes and systems must support those objectives. 


«The LMD must endorse the activity if it is to be successful. It should be initiated and 
managed by Headquarters, but must have regional participation and buy-in. 


* The next step is to develop a set of business processes that support the Logistics strategic 
objectives and satisfy specific business function needs. The processes should be organized by 
business function, not by system. One business process may incorporate the use of several 
systems. 


* Тһе Assessment team has provided a head start for FEMA, by documenting the “as is” 
process state. The next step is to determine the “to be” state for the processes. Although the 
ТАУ program office can drive the process development, this activity will require 
participation and input from FEMA personnel at Headquarters, the DCs, and the Regions, 


* The process development should occur in iterative releases. This will enable FEMA to benefit 
from small wins over time. 


* The processes must be rolled out to the user community through a solid communication and 
training initiative. A key success factor for communications and training will be obtaining the 
relevant leadership levels of involvement to reinforce the importance of following processes. 


* The most important factor in sustaining and improving the processes over time is to monitor 
and measure performance of the processes. Data should be collected, analyzed and results 
communicated throughout the organization. 


* ТАУ needs to move to a model where the overall business process е an integral 
component of the systems development life cycle. Requirements for system enhancements 
should be managed across the TAV program, instead of by individual system teams. Some 
requirements may be addressed through process modifications, instead of system changes. In 
addition, system changes usually impact processes, so processes must be considered. 


Current asset visibility is very limited, but could be expanded via new processes 


The Assessment team identified several instances where visibility of orders and assets could be 
improved with alterations to existing processes and minimal system configuration changes. A 
visibility matrix comprised of order type with corresponding visibility level is found in Appendix 
D of this assessment. While 100% systematic real-time visibility of the entire supply chain is a 
long-term objective, FEMA should focus on improving visibility over time. By creating a 
roadmap and improving visibility in stages, the organization can accomplish short-term wins and 
make a dramatic improvement in the visibility of the supply chain. 


Figure 4 provides a graphical depiction of the visibility matrix. The green lines indicate areas 
where FEMA currently has in-transit visibility; the red lines indicate lack of in-transit visibility. 
The yellow numbers apply to Table 18, which provides recommendations to improve visibility. 
The figure shows that FEMA only has in-transit visibility, i.e., "dots on the map,” for shipments 
originating from DCs in Regions 4 or 6, and being transported via FEMA organic or leased 
trailers. FEMA also has visibility of DLA shipments of Meals Ready to Eat (MREs), because 
DLA has a pre-existing interface to IRRIS, which FEMA can leverage. 


A separate issue, not evident in Figure 4, is that for the systems to reflect that a shipment reached 
the FOSA, someone must indicate its receipt in TPM. Currently, FOSA receipt is sporadic, at 
best. As a result, the TRACC personnel must monitor shipments and receive them in the system, 
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after the fact — losing FEMA’s ability to know, in real time, when shipments reach their 
destination. There are many reasons for lack of FOSA receipt, including lack of trained personnel 
and slow TPM response time; however, the Assessment team believes that no one has ever 
focused on a solid process to ensure all shipments are received in real-time. 


Figure 4 - In-transit Visibility 


Vendors 


DC 
(outside 
4 and 6) 


Charitable 
Organizations, 
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Similarly, there needs to be a process for tracking material to the POD and recording when the 
material is received at the POD. This is especially difficult since FEMA normally does not have 
staff at the POD. The eFOSA approach is based on phone calls between the FOSA and a point of 
contact at each POD. 

Table 18 provides recommendations for FEMA to increase visibility through processes. The left 
column lists the areas where FEMA lacks visibility; the numbers correspond to the yellow 
numbers in Figure 4, The right column provides the recommendations. 
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Table 18 - Visibility Improvement Recommendations 


ecommended Process Modificatio! 

These steps are currently in progre: 

* Modify contractual language for commercial truck contracts to require 
placement of FEMA transponder on the trailer. 

* Develop a set of processes for DCs to affix transponders on commercial 
vehicles and recover the transponders after delivery (this process has 
taken over 3 months to develop, and is not complete at the time of this 
Assessment.) 

| Z Lack of real-time receipt * Ensure adequate trained personnel to receive shipments at FOSAs. 

of shipments at FOSAs — | « Implement a process whereby TRACC monitors shipments as they leave 

within Regions 4 and 6 the DC — TRACC would work with the region to identify if there is 

someone to receive the shipment at the FOSA. If not, then TRACC will 
identify a contact at the FOSA to call in the receipt so TRACC can 
update in real time. 

| B Lack of in-transit visibility | * Train DCs to use TPM to create shipments and to affix transponders to 


when the DC is not in trucks leaving from their DC. 
regions 4 and 6 


for commercial truck: 


Lack of real time receipt at 
FOSAs outside of regions 
4and 6 


* [mplement processes with each of the regions to ensure staffing at 
FOSAs to receive shipments. 

= Lack of visibility of The best solution to this problem is to utilize the capabilities inherent in the 

vendor shipments TPM software to integrate with vendor systems. This functionality allows 

vendors to automatically receive FEMA orders and to send FEMA 
shipment information electronically. Until that functionality is 
implemented, the following processes changes are recommended: 

= Commodity Managers or the TRACC follow up with vendors to 
determine when they are shipping commodities. 

* Commodity Manager or the TRACC create a shipment in TPM indicating 
the type of commodity and the time of the shipment. 

* TPM shipment information is automatically sent to IRRIS. 

* DCs, FOSAs, DHS/FEMA can query IRRIS to identify all shipments. 
However, vendor shipments will not be visible on the IRRIS map, only 
on the IRRIS list screen, because there are no FEMA transponders on the 
trucks. 

The above recommendations assume that the orders to vendors are 

executed by the Headquarters Commodity Managers; however Regions can 

order from vendors without going through Headquarters. Therefore, the 
following recommendation is also included: 

* Implement TPM Order and Shipment processes within the Regions to 
ensure a common process and visibility, regardless of who is placing the 
order. 

| 6 Lack of visibility of OFA | Same recommended solution as above for vendors 

shipments 
Lack of visibility of The same solution is recommended as above for vendors, but only if the 

charitable organization organizations work with Logistics to ship the commodities. If they come 

shipments unannounced, not much can be done to track their shipments. FEMA can 
ask the charitable organizations to notify Logistics before shipments are 
sent to FEMA DCs or FOSAs. 

[8 Lack of visibility of FOSA | This problem requires a multi-part solution: 


FEMA TAV Systems Solution Assessment v.1.1 Page 77 
October 15, 2007 
-FOR OFFICIAL USE-ONEY 


СОЛИ 
7 ч 


е 


JŠ FEMA Prepare. Respond. Deliver. 


Сур se Total Asset Visibility 


Visibility Deficiency | Recommended Process Modification 
shipments to the POD * Create TPM shipments for movements from the FOSA to the POD (either 
directly or by using eFOSA shipments.) 
” Affix transponders to trailers moving from the FOSA to the POD, that 
did not arrive at the FOSA with a transponder. 


* Determine receipt at the POD by: 


* Calling points of contact at the POD; or 

= Leveraging USACE’s staff at the PODS to receive the shipment in 
TPM; or 

* Using geo tracking to identify when the shipment is in the physical 
location of the POD and generating an automatic receipt. 


Develop processes for reporting during an event to maintain accurate situational 
awareness 


The Assessment team observed that reporting during an event is fragmented among systems. No 
one system provides all of the information needed, and there is no central repository or reporting 
hub to provide consistent data efficiently during an event. Furthermore, the TAV and eTasker 
systems only address orders that come through Headquarters. The Regions can also place orders 
directly to vendors or OFAs — which are not captured їп ТАУ systems. 


There are several different systems, including spreadsheets, which provide key information, and 
multiple systems report on the same information. Depending on the timeframe in which the 
systems are updated, they may provide different answers to the same question, Furthermore, the 
Assessment team observed that the Division within Logistics that is responsible for reporting is 
not using reports available from ТАУ. Although the reporting tools are less than ideal, the 
situation could be mitigated with a set of processes that ensure consistent information across 
FEMA. This is lacking in FEMA today. The reporting issues result in the following impacts 
during an event: 


* Field representatives complain that they are barraged with phone calls. Headquarters and 
Regional management and staff call the FOSAs to ask about status of shipments coming in 
and out of the staging areas. FEMA has not instantiated processes to use its systems to report 
status during an event. The reasons include lack of confidence in the systems, fragmentation 
of data across multiple systems, lack of awareness systems, and lack of training. As a result, 
the best source of information is “on the ground.” This creates problems for the field. 


* When multiple sources provide different answers to the same question, FEMA’s credibility 
suffers. The reasons may be perfectly justified — including different business rules or update 
times — but the result is lack of confidence in FEMA’s ability to maintain situational 
awareness. As a result, FEMA management asks for additional reports and calculations and 
calls to the field are increased. 


Table 19 lists the various systems and spreadsheets that provide key information about FEMA’s 
supply chain. The table shows the fragmentation and duplication of information. It also identifies 
areas where the systems do not track the full breadth of information required for full asset 
visibility. 

Table 19 - FEMA Logistics Reporting 


Information Coverage 
* Requests for Assets. | Includes: 


eTasker 

* All requests for commodities that are fulfilled via FEMA 
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Tool Coverage 
Headquarters 
Does not include: 
* Requests that are fulfilled by the Regions 
TPM * Orders Includes: 
* Shipments * Orders that are fulfilled through FEMA Headquarters 
* [nventory "= Shipments originating from DCs within Regions 4 and 6 
NOTE: Reporting via | = Inventory at Atlanta and Fort Worth DCs and Region 4 and 6 
online screens — no regional warehouses 
hard copy report Does not Include: 


capability until the * Orders that are fulfilled by the Regions 

Peformance Мойше - a Shipments for commodities originating from vendors, OFAs, 

is.implemented. and charitable organizations 

* Inventory outside of Regions 4 and 6 

IRRIS * Shipments, Includes: 

* Major assets * Shipments originating from a DC in Regions 4 and 6 

* In-transit Visibility * In-transit shipments originating from a DC, with a transponder 
(i.e., FEMA leased or organic transportation only) 

* DLA shipments 

* FEMA assets such as generators that have a GPS transponder 
attached 

Does not Include: 

* Shipments for commodities originating from vendors, OFAs, 
and charitable organizations 

* In-transit visibility for shipments from DCs shipped via 
commercial carriers 

" Shipments from FOSAs to PODs 

* Shipments outside of Regions 4 and 6 


DC * Inventory Includes: 

Spreadsheets | " Shipments * Inventory at each of the DCs — sent to Headquarters once a day 
(twice a day during disasters) 

* Incoming and outgoing shipments — sent to Headquarters once a 
day (twice a day during disasters) 

Does not Include: 

* Real-time updates at Headquarters 


FOSA * FOSA Receipts | Includes: 

Spreadsheets | * FOSA Shipments = Receipts and shipments for an individual FOSA 

Does not Include: 

* Information across FOSAs 

* Consistent method of FOSA reporting; each Region uses its own 
method for FOSA tracking/reporting. 


Tn the short term, the Assessment team recommends implementing a comprehensive process- 
driven approach to reporting before next year's hurricane season. FEMA Logistics should form a 
cross-functional team to review all the sources for reports and the different methods for reporting. 
The team should determine all the key information that is needed during an event and develop 
processes for acquiring the data quickly and to ensure that there is only one official report or 
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answer. FEMA should then communicate, instantiate, and enforce the processes. Potential options 
include: 


= Development of a consolidated report from the different systems, posted on the Intranet 
periodically; and 

* Designated reporting group for ad hoc questions -- e.g., the LMC or the TRACC; those 
groups would be trained in use of the systems and would use standardized processes for 
running reports. 

In the long-term, FEMA should work on improving breadth and accuracy of the data within the 

systems, and investigate a data warehousing alternative to provide consolidated reporting. 


Improve the training process by taking an integrated approach across all TAV 
systems 


There is currently no cohesive approach to ТАУ training. Each system is trained individually and 
there is no standardized process for delivering a comprehensive overview of the TAV system. To 
increase system usage and user acceptance of all systems that comprise the TAV solution, the 
training approach should require that each system train toward the same overall business 
processes and provide context for how each system fits into the overall ТАУ picture. 


System usage is critical to meeting FEMA’s business objective of achieving real-time visibility 
into requests, orders, shipments and deliveries. To achieve this business objective, FEMA needs 
to have a scalable workforce that is trained in both system usage and business processes to 
improve response efforts at any level. To help increase user adoption and proper system usage, 
the following recommendations should be considered to improve training: 


* Establish standardized processes for managing the training process and a comprehensive 
ТАУ training approach that cuts across the systems. 

* Designate a ТАУ Training Manager, or integrate ТАУ in the overall training approach for the 
LMD, to ensure a consistent approach for training. 

* Training needs to extend beyond FEMA Headquarters and full-time personnel. DAEs must 
also be trained in both process and system usage to ensure scalability. 

«Emphasize FEMA business processes during training efforts and demonstrate how the system 
supports those processes. 

«Take a role-based approach to training so that each person understands their responsibility 
and level of accountability throughout FEMA's supply chain for following correct processes 
and system usage. 

*" Use scenario-based examples in training to provide proper operational context ог 
methodology for all FEMA personnel. 

* Introduce knowledge management tools and training practices to reinforce training. Examples 
of tools could include online guides, role-specific cheat sheets or reference guides, or 
refresher training courses (online or in-person). 
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6.5 Warehouse Management 


A significant component of the FEMA ТАУ investment is for the Manhattan WM software. It is 
licensed separately from the TPM module. The software is currently installed in only two of the 
distribution centers — Fort Worth and Atlanta — and FEMA is using minimal WM functionality. 


Although FEMA is investigating 3PL providers for warehousing, it will most likely maintain some 
level of organic warehouses in the future. Furthermore, even a partial 3PL solution is over one year 
away. Even if FEMA decides to outsource the management of its organic warehouses, the provider 
will require specialized software, which at this point would most likely be the Manhattan WM 
product, 


The management of FEMA warehouses can be optimized by proper configuration and use of the 
WM software. In addition, if implemented properly, the software can help FEMA make best use of 
its temporary workforce, by automating decision processes that currently burden the very limited 
number of experienced FEMA employees available during disaster situations. For these reasons, the 
Assessment team recommends that FEMA more fully integrate the Manhattan software into its 
warehouses and adopt the supply chain best practices supported by the software. 


(Additional warehouse management observations and recommendations that are not directly related 
to the warehouse software are discussed in Appendix C.) 


Appropriate use of the WM software will help optimize warehouse receiving 


Proper use of automated due-ins and Advance Shipping Notices (ASNs) will speed the receiving 
process. Using the system to determine where to store items maximizes the efficiency of the 
operation. Today, in the DCs, the warehouse depends upon the IMS or the supervisor to 
determine where to store each pallet. This process is an inefficient use of management and 
warehouse talent. To use the directed put-away properly, the DC would have to input the business 
rules it wishes to follow. 


The Fort Worth DC should start updating the location information in the system; Atlanta should 
wait until a replacement site is identified and the warehouse racking plan is firm. Establishing the 
distance from receiving to each of the locations and updating the system with location dimensions 
and capacity will take time. Typically, this effort is completed as a part of the initial set-up or 
prior to moving into a different facility. It is a one-time effort that is modified only when 
locations are added or removed, or racking is reconfigured—neither situation occurs frequently. 


DCs with JFO kits should manage all components of the JFO as kits in the 
Manhattan warehouse management system. 


Today, Fort Worth and Atlanta use the system to manage portions of the JFO kits, but the 
facilities have not bundled all components of a JFO kit as a single unit. The WM system has the 
ability to manage the entire complement of items required for a kit, including multiple levels of 
nested kits. Using the kitting functions enables the warehouse to designate specific components of 
the kit, and ensures that the proper components are selected without supervisory oversight. It also 
provides leadership with the tools to know the status of a kit’s components. 


JFO kits are currently not configured to take advantage of the nested kitting capabilities of the 
Manhattan software. All the contents of the kit are stored in adjacent locations of an aisle. 
Warehouse leadership can walk an aisle and get a sense of the status of the kit, but the system 
does not provide a status of the complete package. 
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The JFO kits are standard pack-up kits used by each JFO as they activate the office. These kits 
accommodate 50 people with a specific list of standard commodities. Each kit is designed to fit 
on a trailer, and the contents maximize the trailer capacity. Using the system to manage all 
components of the kit reduces the supervisory oversight required when pulling the kit and 
provides management with visibility of the status of all kits. 


If the facility uses the system directed picking as it is currently without nested kitting, the various 
components could be pulled from locations throughout the warehouse, effectively breaking up a 
kit. Consequently, when the facility needs to pull a kit, leadership determines where to pull the 
material, supervises the temporary warehouse worker pulling the order to ensure the correct item 
is pulled, and updates the system after the items have been selected. 

As shown in Figure 5, two levels of kits comprise the JFO kit. The Admin Kit is a parent and 
includes all the small supplies required to support an office. The Admin Kit is also a child to the 
Standard JFO Kit along with the other components of the kit, including the folding tables and 
pallet jack. Nested kitting is a standard practice in world class warehouse operations. 


Figure 5 - Nested Kitting for JFO Kits 
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DCs need to have designated super users for the Manhattan software to 
understand the system and adjust transactions while maintaining data integrity 


The team noted that DCs did not have any “super users” with sufficient rights in the warehouse 
software to resolve problems or address special circumstances. Instead, this level of user rights is 
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restricted to the ASB system administration staff, or, in some cases, the TRACC — both of which 
are far from the operational front lines. For example while visiting Fort Worth the team 
discovered that GPS units which failed after assignment and required replacement, or changes to 
orders once loaded on a truck, resulted in the warehouse reversing the entire order, A super user 
at the DC could have adjusted the order without having to reverse the entire transaction. Limiting 
the super user position to a few experienced warehouse team members would ensure that the 
super users understand the impact of their changes to the warehouse operations and that WMS, 
and systems controls remain in place. 


DCs need to use the Manhattan warehouse management system directed picking, 
which would increase the efficiency of limited staffing during disasters 


Today, the warehouse team uses a manually directed picking process and manually updates the 
system. The system requires that the pick plan be added in the business rules set-up and it can be 
tailored to accommodate Z and U picking! throughout the facility. As an example, setting the 
picking plan would enable Fort Worth to store water and other commodities on the floor as they 
do in Building 12 and pick it from the floor locations as part of the normal picking scheme. 
Again, properly setting up the business rules available in the system will allow the system to 
function properly. For instance, if a pallet is located in a Z location le) the system will not 
direct a pick from the rack behind the aisle location. 


!! In a Z pick the warehouse worker alternates between sides of the aisle when selecting an order—the 
system is set-up to pick slot 151, then 152, then 153, etc. In a U pick the worker selects all the order from 
one side of the aisle, makes a U-turn and picks all the orders from the other side of the same aisle; slot 151, 
153, 155, 157 before turning around and picking 158, 156, 154, 152, etc. A Z pick works well when the 
aisles width is narrow because the worker doesn't have to walk far. A U pick works better in wider aisles 
where the worker has room to maneuver the material handling equipment. While the U pick accommodates 
multiple selectors in an aisle easier than a Z pick, the operator has to forgo valuable storage real estate to 
support the wider aisles. Most WMSs are designed to support both types of picking, even within the same 
facility. This provides flexibility for the warehouse operator to accommodate limitations of the physical 
plant. Because of the location of columns, some DCs will have aisles that vary in width and will use both 
types of picking dependent on the specific aisle. 
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